You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@cloudstack.apache.org by 조대형 <ca...@renet.kr> on 2017/01/18 07:47:24 UTC
Cloudstack 4.7 Secondary Storage problem
Hi, all
I am facing secondary storage problem.
I am using NFS Server to use primary and secondary storage.
After rebooting the host,
In Dashboard Cloudstack , it says Secondary Storage 0.00 KB / 0.00 KB
Primary Storage is okay.
After all, system VMs are not up. It was working fine before rebooting.
I have tried several times of restart of Management service.
Any ideas?
Thanks,
Re: Cloudstack 4.7 Secondary Storage problem
Posted by Dag Sonstebo <Da...@shapeblue.com>.
Hi Carlcho,
Sorry appreciate we’re running two different problems in the same email thread at the moment.
In your case I think we need to understand a bit more about your problem – is this a new CloudStack install? Did you seed your system VM templates in secondary storage before adding this?
You say you’ve deleted secondary storage – do you mean the SSVM or do you mean the secondary storage pool? If you have deleted your secondary storage pool you need to add this back in.
Please also note "Unable to start a VM due to insufficient capacity" is a very common error in CloudStack – and we would have to see your management logs to understand the root cause of this.
Regards,
Dag Sonstebo
Cloud Architect
ShapeBlue
On 18/01/2017, 12:28, "조대형" <ca...@renet.kr> wrote:
Even though I deleted the second storage, it shows Secondary Storage 0.00 KB / 0.00 KB".
So, I can't start any VMs in my cloudstack.
When I start VM, it says "Unable to start a VM due to insufficient capacity"
Any ideas?
-----Original Message-----
From: Paul Angus [mailto:paul.angus@shapeblue.com]
Sent: Wednesday, January 18, 2017 5:55 PM
To: users@cloudstack.apache.org
Subject: RE: Cloudstack 4.7 Secondary Storage problem
SSVM reports the secondary storage space, but doesn't run from secondary storage.
you're seeing "Secondary Storage 0.00 KB / 0.00 KB" because the SSVM isn't running.
If you can't start the SSVM again, then the host(s) likely have the stale NFS issue. If no VMs are running on the host that had the SSVM on it, then try restarting that. Otherwise you need to look at the logs on that host.
Kind regards,
Paul Angus
paul.angus@shapeblue.com
www.shapeblue.com
53 Chandos Place, Covent Garden, London WC2N 4HSUK @shapeblue
-----Original Message-----
From: 조대형 [mailto:carlcho@renet.kr]
Sent: 18 January 2017 08:48
To: users@cloudstack.apache.org
Subject: RE: Cloudstack 4.7 Secondary Storage problem
There is no space on Secodary Storage.
Secondary Storage 0.00 KB / 0.00 KB
SO, SSVM is not up.
-----Original Message-----
From: Paul Angus [mailto:paul.angus@shapeblue.com]
Sent: Wednesday, January 18, 2017 5:37 PM
To: users@cloudstack.apache.org
Subject: RE: Cloudstack 4.7 Secondary Storage problem
Hi there,
Chances are that you have stale NFS sessions on the Secondary Storage VM(s).
Try rebooting the SSVM(s)
If that doesn't clear the problem, can you run /usr/local/cloud/systemvm/ssvm-check.sh on the SSVM and report back the results.
Kind regards,
Paul Angus
paul.angus@shapeblue.com
www.shapeblue.com
53 Chandos Place, Covent Garden, London WC2N 4HSUK @shapeblue
-----Original Message-----
From: sachin patil [mailto:sachin931350@gmail.com]
Sent: 18 January 2017 08:07
To: users@cloudstack.apache.org
Subject: Re: Cloudstack 4.7 Secondary Storage problem
Can you please share your management_server logs
On Tue, Jan 17, 2017 at 11:47 PM, 조대형 <ca...@renet.kr> wrote:
> Hi, all
>
> I am facing secondary storage problem.
>
> I am using NFS Server to use primary and secondary storage.
>
> After rebooting the host,
>
> In Dashboard Cloudstack , it says Secondary Storage 0.00 KB / 0.00 KB
>
> Primary Storage is okay.
>
>
>
> After all, system VMs are not up. It was working fine before rebooting.
>
>
>
> I have tried several times of restart of Management service.
>
>
>
> Any ideas?
>
>
>
> Thanks,
>
>
Dag.Sonstebo@shapeblue.com
www.shapeblue.com
53 Chandos Place, Covent Garden, London WC2N 4HSUK
@shapeblue
RE: Cloudstack 4.7 Secondary Storage problem
Posted by 조대형 <ca...@renet.kr>.
Even though I deleted the second storage, it shows Secondary Storage 0.00 KB / 0.00 KB".
So, I can't start any VMs in my cloudstack.
When I start VM, it says "Unable to start a VM due to insufficient capacity"
Any ideas?
-----Original Message-----
From: Paul Angus [mailto:paul.angus@shapeblue.com]
Sent: Wednesday, January 18, 2017 5:55 PM
To: users@cloudstack.apache.org
Subject: RE: Cloudstack 4.7 Secondary Storage problem
SSVM reports the secondary storage space, but doesn't run from secondary storage.
you're seeing "Secondary Storage 0.00 KB / 0.00 KB" because the SSVM isn't running.
If you can't start the SSVM again, then the host(s) likely have the stale NFS issue. If no VMs are running on the host that had the SSVM on it, then try restarting that. Otherwise you need to look at the logs on that host.
Kind regards,
Paul Angus
paul.angus@shapeblue.com
www.shapeblue.com
53 Chandos Place, Covent Garden, London WC2N 4HSUK @shapeblue
-----Original Message-----
From: 조대형 [mailto:carlcho@renet.kr]
Sent: 18 January 2017 08:48
To: users@cloudstack.apache.org
Subject: RE: Cloudstack 4.7 Secondary Storage problem
There is no space on Secodary Storage.
Secondary Storage 0.00 KB / 0.00 KB
SO, SSVM is not up.
-----Original Message-----
From: Paul Angus [mailto:paul.angus@shapeblue.com]
Sent: Wednesday, January 18, 2017 5:37 PM
To: users@cloudstack.apache.org
Subject: RE: Cloudstack 4.7 Secondary Storage problem
Hi there,
Chances are that you have stale NFS sessions on the Secondary Storage VM(s).
Try rebooting the SSVM(s)
If that doesn't clear the problem, can you run /usr/local/cloud/systemvm/ssvm-check.sh on the SSVM and report back the results.
Kind regards,
Paul Angus
paul.angus@shapeblue.com
www.shapeblue.com
53 Chandos Place, Covent Garden, London WC2N 4HSUK @shapeblue
-----Original Message-----
From: sachin patil [mailto:sachin931350@gmail.com]
Sent: 18 January 2017 08:07
To: users@cloudstack.apache.org
Subject: Re: Cloudstack 4.7 Secondary Storage problem
Can you please share your management_server logs
On Tue, Jan 17, 2017 at 11:47 PM, 조대형 <ca...@renet.kr> wrote:
> Hi, all
>
> I am facing secondary storage problem.
>
> I am using NFS Server to use primary and secondary storage.
>
> After rebooting the host,
>
> In Dashboard Cloudstack , it says Secondary Storage 0.00 KB / 0.00 KB
>
> Primary Storage is okay.
>
>
>
> After all, system VMs are not up. It was working fine before rebooting.
>
>
>
> I have tried several times of restart of Management service.
>
>
>
> Any ideas?
>
>
>
> Thanks,
>
>
RE: Cloudstack 4.7 Secondary Storage problem
Posted by Paul Angus <pa...@shapeblue.com>.
SSVM reports the secondary storage space, but doesn't run from secondary storage.
you're seeing "Secondary Storage 0.00 KB / 0.00 KB" because the SSVM isn't running.
If you can't start the SSVM again, then the host(s) likely have the stale NFS issue. If no VMs are running on the host that had the SSVM on it, then try restarting that. Otherwise you need to look at the logs on that host.
Kind regards,
Paul Angus
paul.angus@shapeblue.com
www.shapeblue.com
53 Chandos Place, Covent Garden, London WC2N 4HSUK
@shapeblue
-----Original Message-----
From: 조대형 [mailto:carlcho@renet.kr]
Sent: 18 January 2017 08:48
To: users@cloudstack.apache.org
Subject: RE: Cloudstack 4.7 Secondary Storage problem
There is no space on Secodary Storage.
Secondary Storage 0.00 KB / 0.00 KB
SO, SSVM is not up.
-----Original Message-----
From: Paul Angus [mailto:paul.angus@shapeblue.com]
Sent: Wednesday, January 18, 2017 5:37 PM
To: users@cloudstack.apache.org
Subject: RE: Cloudstack 4.7 Secondary Storage problem
Hi there,
Chances are that you have stale NFS sessions on the Secondary Storage VM(s).
Try rebooting the SSVM(s)
If that doesn't clear the problem, can you run /usr/local/cloud/systemvm/ssvm-check.sh on the SSVM and report back the results.
Kind regards,
Paul Angus
paul.angus@shapeblue.com
www.shapeblue.com
53 Chandos Place, Covent Garden, London WC2N 4HSUK @shapeblue
-----Original Message-----
From: sachin patil [mailto:sachin931350@gmail.com]
Sent: 18 January 2017 08:07
To: users@cloudstack.apache.org
Subject: Re: Cloudstack 4.7 Secondary Storage problem
Can you please share your management_server logs
On Tue, Jan 17, 2017 at 11:47 PM, 조대형 <ca...@renet.kr> wrote:
> Hi, all
>
> I am facing secondary storage problem.
>
> I am using NFS Server to use primary and secondary storage.
>
> After rebooting the host,
>
> In Dashboard Cloudstack , it says Secondary Storage 0.00 KB / 0.00 KB
>
> Primary Storage is okay.
>
>
>
> After all, system VMs are not up. It was working fine before rebooting.
>
>
>
> I have tried several times of restart of Management service.
>
>
>
> Any ideas?
>
>
>
> Thanks,
>
>
RE: Cloudstack 4.7 Secondary Storage problem
Posted by 조대형 <ca...@renet.kr>.
There is no space on Secodary Storage.
Secondary Storage 0.00 KB / 0.00 KB
SO, SSVM is not up.
-----Original Message-----
From: Paul Angus [mailto:paul.angus@shapeblue.com]
Sent: Wednesday, January 18, 2017 5:37 PM
To: users@cloudstack.apache.org
Subject: RE: Cloudstack 4.7 Secondary Storage problem
Hi there,
Chances are that you have stale NFS sessions on the Secondary Storage VM(s).
Try rebooting the SSVM(s)
If that doesn't clear the problem, can you run /usr/local/cloud/systemvm/ssvm-check.sh on the SSVM and report back the results.
Kind regards,
Paul Angus
paul.angus@shapeblue.com
www.shapeblue.com
53 Chandos Place, Covent Garden, London WC2N 4HSUK @shapeblue
-----Original Message-----
From: sachin patil [mailto:sachin931350@gmail.com]
Sent: 18 January 2017 08:07
To: users@cloudstack.apache.org
Subject: Re: Cloudstack 4.7 Secondary Storage problem
Can you please share your management_server logs
On Tue, Jan 17, 2017 at 11:47 PM, 조대형 <ca...@renet.kr> wrote:
> Hi, all
>
> I am facing secondary storage problem.
>
> I am using NFS Server to use primary and secondary storage.
>
> After rebooting the host,
>
> In Dashboard Cloudstack , it says Secondary Storage 0.00 KB / 0.00 KB
>
> Primary Storage is okay.
>
>
>
> After all, system VMs are not up. It was working fine before rebooting.
>
>
>
> I have tried several times of restart of Management service.
>
>
>
> Any ideas?
>
>
>
> Thanks,
>
>
RE: Cloudstack 4.7 Secondary Storage problem
Posted by Paul Angus <pa...@shapeblue.com>.
Hi there,
Chances are that you have stale NFS sessions on the Secondary Storage VM(s).
Try rebooting the SSVM(s)
If that doesn't clear the problem, can you run /usr/local/cloud/systemvm/ssvm-check.sh on the SSVM and report back the results.
Kind regards,
Paul Angus
paul.angus@shapeblue.com
www.shapeblue.com
53 Chandos Place, Covent Garden, London WC2N 4HSUK
@shapeblue
-----Original Message-----
From: sachin patil [mailto:sachin931350@gmail.com]
Sent: 18 January 2017 08:07
To: users@cloudstack.apache.org
Subject: Re: Cloudstack 4.7 Secondary Storage problem
Can you please share your management_server logs
On Tue, Jan 17, 2017 at 11:47 PM, 조대형 <ca...@renet.kr> wrote:
> Hi, all
>
> I am facing secondary storage problem.
>
> I am using NFS Server to use primary and secondary storage.
>
> After rebooting the host,
>
> In Dashboard Cloudstack , it says Secondary Storage 0.00 KB / 0.00 KB
>
> Primary Storage is okay.
>
>
>
> After all, system VMs are not up. It was working fine before rebooting.
>
>
>
> I have tried several times of restart of Management service.
>
>
>
> Any ideas?
>
>
>
> Thanks,
>
>
Re: Cloudstack 4.7 Secondary Storage problem
Posted by me...@students.iitmandi.ac.in.
Hi,
So I was able to fix the error (Secondary storage shows 0 in dashboard). I
simply logged into the ssvm and ran the health checks, which said NFS
currently not mounted. So I flushed the iptables on the host of my
cloudstack setup( which also happens to be the NFS server and Management
server) and then retried the command "mount" in ssvm and it had mounted.
So I restarted the services and now I am able to view the Secondary
Storage 6.33GB/50GB.
However, When I try too download and ISO or upload. It fails.
When I try to download the default XS-tools.iso it gives me an error
saying "Unable to extract the ISO xs-tools.iso as it resides on host and
not on SSVM"
If I try to upload any iso,it again fails and in the status there is
nothing that is updated.
When I checked the ssvm troubleshooting steps
It read that I need to fix up my secstorage.allowed.internal.sites.
Now the problem is that I have given it the ip of my host(which also acts
as a Management server,NFS server and Mysql Database) : -10.8.19.111/32
While the range of ips I gave to Pods and guest VMs was 10.8.170.x range.
Which means that even ssvm has an IP in that range.
What should I do?
> Merlin,
>
> Can you actually ping 10.8.19.111? Have you set up the NFS shares
> according to CloudStack specification ( /export
> <CIDR>(rw,async,no_root_squash,no_subtree_check) )? What is the contents
> of your /etc/exports file?
>
> In short it looks like you have either a connectivity or permissions issue
> \u2013 which is unrelated to CloudStack. Until you have this sorted you
> won\u2019t be able to spin up system VMs etc. With regards to ACLs on your
> NFS share \u2013 keep in mind you need to open this up to any IP address in
> your management IP range \u2013 since the SSVM can appear on any IP in this.
>
> Regards,
> Dag Sonstebo
> Cloud Architect
> ShapeBlue
>
> On 18/01/2017, 12:10, "merlin@students.iitmandi.ac.in"
> <me...@students.iitmandi.ac.in> wrote:
>
> It did not throw me any error this time, however, there was no
> response
> either. The cursor just kept blinking for a very long time. untill I
> did
> ctrl+c.
>
> Also, a brief observation: when I write mount in my nfs server:
>
> I see three lines at the end
> 10.8.19.111:/export/primary on
> /mnt/38454858-08bb-3e92-88c9-354465ad3715
> type nfs
> (rw,noac,actimeo=0,vers=4,addr=10.8.19.111,clientaddr=10.8.19.111)
> 10.8.19.111:/export/primary on /mnt/primary type nfs
> (rw,rsize=8192,wsize=8192,timeo=14,intr,vers=3,noac,actimeo=0,addr=10.8.19.111)
> 10.8.19.111:/export/secondary on /mnt/secondary type nfs
> (rw,rsize=8192,wsize=8192,timeo=14,intr,vers=3,noac,actimeo=0,addr=10.8.19.111)
>
> Which used to be just 2 when I was using cloudstack version 4.4. I
> recently installed 4.8 and I see this after clean installation.
>
> > try "mount -t nfs 10.8.19.111:/export/secondary/ /tmp/secondary"
> >
> > *Vivek Kumar*
> > Virtualization and Cloud Consultant
> >
> > [image: http://www.indiqus.com/images/logo.jpg]
> <http://www.indiqus.com/>
> > *I*ndi*Q*us Technologies Pvt Ltd
> > A-98, LGF, C.R.Park, New Delhi - 110019
> > *O* +91 11 4055 1411 | *M* +91 7503460090
> > www.indiqus.com <http://www.indiqus.com/>
> >
> > On Wed, Jan 18, 2017 at 5:23 PM, <me...@students.iitmandi.ac.in>
> wrote:
> >
> >> Hi,
> >>
> >> I tried like you said: here is the result
> >>
> >> root@s-2-VM:~# mount -t NFS 10.8.19.111:/export/secondary/
> >> /tmp/secondary/
> >> mount: unknown filesystem type 'NFS'
> >> root@s-2-VM:~# mount -t 10.8.19.111:/export/secondary/
> /tmp/secondary/
> >> nfs
> >> mount: mount point nfs does not exist
> >>
> >> I might be running it wrong. Also, i checked the ssvm for the nfs
> >> package
> >> and neither was it there, nor could i download.
> >> Please help
> >>
> >>
> >> > Hi Merlin,
> >> >
> >> > What Paul means is you need to log on to the SSVM, then do
> something
> >> like
> >> > this:
> >> >
> >> > mkdir /tmp/secondary
> >> > mount â\u20ac\u201ct NFS <NFS server IP>:/<NFS/path/here>/
> /tmp/secondary
> >> >
> >> > This will tell you if your NFS permissions are configured
> correctly.
> >> >
> >> > Regards,
> >> > Dag Sonstebo
> >> > Cloud Architect
> >> > ShapeBlue
> >> >
> >> > On 18/01/2017, 09:29, "merlin@students.iitmandi.ac.in"
> >> > <me...@students.iitmandi.ac.in> wrote:
> >> >
> >> > Also when you say mount from ssvm. Does it mean this?
> >> > Log into ssvm and run "mount":- the reply is down below. I am
> not
> >> able
> >> > to
> >> > understand much even from the documentation,
> >> > I am sorry for the lack of full knowledge in this concern.
> >> >
> >> >
> >> > sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
> >> > proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
> >> > udev on /dev type devtmpfs
> >> > (rw,relatime,size=10240k,nr_inodes=62128,mode=755)
> >> > devpts on /dev/pts type devpts
> >> > (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
> >> > tmpfs on /run type tmpfs
> >> > (rw,nosuid,noexec,relatime,size=50896k,mode=755)
> >> > /dev/disk/by-uuid/30c81d3d-ee9f-4a88-81c1-5f349b22ba1d on /
> type
> >> ext4
> >> > (rw,relatime,errors=remount-ro,user_xattr,barrier=1,data=ordered)
> >> > tmpfs on /run/lock type tmpfs
> >> > (rw,nosuid,nodev,noexec,relatime,size=5120k)
> >> > tmpfs on /run/shm type tmpfs
> >> > (rw,nosuid,nodev,noexec,relatime,size=211960k)
> >> > /dev/vda1 on /boot type ext4
> >> > (rw,relatime,user_xattr,barrier=1,data=ordered)
> >> > /dev/vda6 on /home type ext4
> >> > (rw,relatime,user_xattr,barrier=1,data=ordered)
> >> > /dev/vda8 on /opt type ext4
> >> > (rw,relatime,user_xattr,barrier=1,data=ordered)
> >> > /dev/vda11 on /tmp type ext4
> >> > (rw,relatime,user_xattr,barrier=1,data=ordered)
> >> > /dev/vda7 on /usr type ext4
> >> > (rw,relatime,user_xattr,barrier=1,data=ordered)
> >> > /dev/vda9 on /var type ext4
> >> > (rw,relatime,user_xattr,barrier=1,data=ordered)
> >> > /dev/vda10 on /var/log type ext4
> >> > (rw,relatime,user_xattr,barrier=1,data=ordered)
> >> > rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs
> >> (rw,relatime)
> >> >
> >> > > Hi Merlin,
> >> > >
> >> > > To troubleshoot, try mounting the NFS yourself from the
> SSVM and
> >> get
> >> > that
> >> > > working.
> >> > >
> >> > > the likely problem is permissions on the NFS mounts. Either
> the
> >> IP
> >> > of the
> >> > > SSVM is not allowed or the permissions don't include
> >> no_root_squash
> >> > (or
> >> > > equivalent) the SSVM is probably connecting as 'cloud', so
> >> > connections
> >> > > have to be mapped to 'root'. Finally the SSVM only
> reliably
> >> > supports
> >> > > NFSv3
> >> > >
> >> > >
> >> > >
> >> > >
> >> > > Kind regards,
> >> > >
> >> > > Paul Angus
> >> > >
> >> > > paul.angus@shapeblue.comÃ\u201a
> >> > > www.shapeblue.com
> >> > > 53 Chandos Place, Covent Garden, London WC2N 4HSUK
> >> > > @shapeblue
> >> > >
> >> > >
> >> > >
> >> > >
> >> > > -----Original Message-----
> >> > > From: merlin@students.iitmandi.ac.in
> >> > > [mailto:merlin@students.iitmandi.ac.in]
> >> > > Sent: 18 January 2017 08:48
> >> > > To: users@cloudstack.apache.org
> >> > > Subject: Re: Cloudstack 4.7 Secondary Storage problem
> >> > >
> >> > > And this is the result of check.sh
> >> > >
> >> > > 2 packets transmitted, 2 packets received, 0% packet loss
> >> > round-trip
> >> > > min/avg/max/stddev = 0.400/0.550/0.701/0.151 ms
> >> > > Good: Can ping DNS server
> >> > > ================================================
> >> > > Good: DNS resolves download.cloud.com
> >> > > ================================================
> >> > > ERROR: NFS is not currently mounted
> >> > > Try manually mounting from inside the VM NFS server is
> 10.8.0.1
> >> > PING
> >> > > 10.8.0.1 (10.8.0.1): 48 data bytes
> >> > > 56 bytes from 10.8.0.1: icmp_seq=0 ttl=64 time=0.456 ms
> >> > > 56 bytes from 10.8.0.1: icmp_seq=1 ttl=64 time=0.708 ms
> >> > > --- 10.8.0.1 ping statistics ---
> >> > > 2 packets transmitted, 2 packets received, 0% packet loss
> >> > round-trip
> >> > > min/avg/max/stddev = 0.456/0.582/0.708/0.126 ms
> >> > > Good: Can ping nfs server
> >> > > ================================================
> >> > > Management server is 10.8.19.111. Checking connectivity.
> >> > > Good: Can connect to management server port 8250
> >> > > ================================================
> >> > > Good: Java process is running
> >> > > ================================================
> >> > > Tests Complete. Look for ERROR or WARNING above.
> >> > >
> >> > >
> >> > > The problem is that while installation I did mount nfs?
> >> > > I dont understand what else to do next
> >> > >
> >> > >
> >> > >
> >> > >> Hi, even I am facing the same problem.. and here are the
> logs
> >> from
> >> > >> today:
> >> > >>
> >> > >>
> >> > >>
> >> > >> 2017-01-18 12:16:46,100 DEBUG
> >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> > >> (RouterStatusMonitor-1:ctx-bc5cb4cd) (logid:0ffac658)
> Found 0
> >> > networks
> >> > >> to update RvR status.
> >> > >> 2017-01-18 12:16:46,200 DEBUG [c.c.a.m.AgentManagerImpl]
> >> > >> (AgentManager-Handler-11:null) (logid:) Ping from
> 2(s-2-VM)
> >> > >> 2017-01-18 12:16:47,521 DEBUG [c.c.a.m.AgentManagerImpl]
> >> > >> (AgentManager-Handler-15:null) (logid:) Ping from
> 3(v-1-VM)
> >> > >> 2017-01-18 12:16:49,834 DEBUG [c.c.a.m.AgentManagerImpl]
> >> > >> (AgentManager-Handler-9:null) (logid:) SeqA 3-438472:
> >> Processing
> >> > Seq
> >> > >> 3-438472: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags:
> 11,
> >> > >> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand"
> >> :{"_proxyVmId":1,
> >> > >> "_loadInfo":"{\n
> >> > >> \"connections\": []\n}","wait":0}}] }
> >> > >> 2017-01-18 12:16:49,887 DEBUG [c.c.a.m.AgentManagerImpl]
> >> > >> (AgentManager-Handler-9:null) (logid:) SeqA 3-438472:
> Sending
> >> Seq
> >> > >> 3-438472: { Ans: , MgmtId: 279278805450954, via: 3, Ver:
> v1,
> >> > Flags:
> >> > >> 100010,
> >> > >> [{"com.cloud.agent.api.AgentControlAnswer":{"result":
> >> true,"wait":0}}]
> >> > >> }
> >> > >> 2017-01-18 12:16:54,390 DEBUG [c.c.h.d.HostDaoImpl]
> >> > >> (ClusteredAgentManager
> >> > >> Timer:ctx-8464e4b3) (logid:1f15c721) Resetting hosts
> suitable
> >> for
> >> > >> reconnect
> >> > >> 2017-01-18 12:16:54,392 DEBUG [c.c.h.d.HostDaoImpl]
> >> > >> (ClusteredAgentManager
> >> > >> Timer:ctx-8464e4b3) (logid:1f15c721) Completed resetting
> hosts
> >> > >> suitable for reconnect
> >> > >> 2017-01-18 12:16:54,392 DEBUG [c.c.h.d.HostDaoImpl]
> >> > >> (ClusteredAgentManager
> >> > >> Timer:ctx-8464e4b3) (logid:1f15c721) Acquiring hosts for
> >> clusters
> >> > >> already owned by this management server
> >> > >> 2017-01-18 12:16:54,393 DEBUG [c.c.h.d.HostDaoImpl]
> >> > >> (ClusteredAgentManager
> >> > >> Timer:ctx-8464e4b3) (logid:1f15c721) Completed acquiring
> hosts
> >> for
> >> > >> clusters already owned by this management server
> >> > >> 2017-01-18 12:16:54,393 DEBUG [c.c.h.d.HostDaoImpl]
> >> > >> (ClusteredAgentManager
> >> > >> Timer:ctx-8464e4b3) (logid:1f15c721) Acquiring hosts for
> >> clusters
> >> > not
> >> > >> owned by any management server
> >> > >> 2017-01-18 12:16:54,394 DEBUG [c.c.h.d.HostDaoImpl]
> >> > >> (ClusteredAgentManager
> >> > >> Timer:ctx-8464e4b3) (logid:1f15c721) Completed acquiring
> hosts
> >> for
> >> > >> clusters not owned by any management server
> >> > >> 2017-01-18 12:16:54,795 DEBUG [c.c.a.m.AgentManagerImpl]
> >> > >> (AgentManager-Handler-5:null) (logid:) SeqA 3-438473:
> >> Processing
> >> > Seq
> >> > >> 3-438473: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags:
> 11,
> >> > >> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand"
> >> :{"_proxyVmId":1,
> >> > >> "_loadInfo":"{\n
> >> > >> \"connections\": []\n}","wait":0}}] }
> >> > >> 2017-01-18 12:16:54,854 DEBUG [c.c.a.m.AgentManagerImpl]
> >> > >> (AgentManager-Handler-5:null) (logid:) SeqA 3-438473:
> Sending
> >> Seq
> >> > >> 3-438473: { Ans: , MgmtId: 279278805450954, via: 3, Ver:
> v1,
> >> > Flags:
> >> > >> 100010,
> >> > >> [{"com.cloud.agent.api.AgentControlAnswer":{"result":
> >> true,"wait":0}}]
> >> > >> }
> >> > >> 2017-01-18 12:16:55,984 INFO
> [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> > >> (AsyncJobMgr-Heartbeat-1:ctx-d0469453) (logid:af5d4f47)
> Begin
> >> > cleanup
> >> > >> expired async-jobs
> >> > >> 2017-01-18 12:16:55,991 INFO
> [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> > >> (AsyncJobMgr-Heartbeat-1:ctx-d0469453) (logid:af5d4f47)
> End
> >> > cleanup
> >> > >> expired async-jobs
> >> > >> 2017-01-18 12:16:59,245 DEBUG [c.c.s.StatsCollector]
> >> > >> (StatsCollector-1:ctx-68069dff) (logid:a7a6974d)
> >> VmStatsCollector
> >> > is
> >> > >> running...
> >> > >> 2017-01-18 12:17:00,841 DEBUG
> >> > [o.a.c.s.SecondaryStorageManagerImpl]
> >> > >> (secstorage-1:ctx-4e101aa5) (logid:65f1f0cb) Zone 1 is
> ready to
> >> > launch
> >> > >> secondary storage VM
> >> > >> 2017-01-18 12:17:01,095 DEBUG
> [c.c.c.ConsoleProxyManagerImpl]
> >> > >> (consoleproxy-1:ctx-393828fa) (logid:7b3d0491) Zone 1 is
> ready
> >> to
> >> > >> launch console proxy
> >> > >> 2017-01-18 12:17:03,283 DEBUG [c.c.s.StatsCollector]
> >> > >> (StatsCollector-3:ctx-cc1f58a1) (logid:3ae41340)
> >> HostStatsCollector
> >> > is
> >> > >> running...
> >> > >> 2017-01-18 12:17:03,343 DEBUG [c.c.a.t.Request]
> >> > >> (StatsCollector-3:ctx-cc1f58a1) (logid:3ae41340) Seq
> >> > >> 1-6227070909769882804: Received: { Ans: , MgmtId:
> >> > 279278805450954,
> >> > >> via:
> >> > >> 1(iuatc), Ver: v1, Flags: 10, { GetHostStatsAnswer } }
> >> > >> 2017-01-18 12:17:04,796 DEBUG [c.c.a.m.AgentManagerImpl]
> >> > >> (AgentManager-Handler-10:null) (logid:) SeqA 3-438474:
> >> Processing
> >> > Seq
> >> > >> 3-438474: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags:
> 11,
> >> > >> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand"
> >> :{"_proxyVmId":1,
> >> > >> "_loadInfo":"{\n
> >> > >> \"connections\": []\n}","wait":0}}] }
> >> > >> 2017-01-18 12:17:04,847 DEBUG [c.c.a.m.AgentManagerImpl]
> >> > >> (AgentManager-Handler-10:null) (logid:) SeqA 3-438474:
> Sending
> >> Seq
> >> > >> 3-438474: { Ans: , MgmtId: 279278805450954, via: 3, Ver:
> v1,
> >> > Flags:
> >> > >> 100010,
> >> > >> [{"com.cloud.agent.api.AgentControlAnswer":{"result":
> >> true,"wait":0}}]
> >> > >> }
> >> > >> 2017-01-18 12:17:05,985 INFO
> [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> > >> (AsyncJobMgr-Heartbeat-1:ctx-7f0de3b8) (logid:e9bbd63a)
> Begin
> >> > cleanup
> >> > >> expired async-jobs
> >> > >> 2017-01-18 12:17:05,991 INFO
> [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> > >> (AsyncJobMgr-Heartbeat-1:ctx-7f0de3b8) (logid:e9bbd63a)
> End
> >> > cleanup
> >> > >> expired async-jobs
> >> > >> 2017-01-18 12:17:14,798 DEBUG [c.c.a.m.AgentManagerImpl]
> >> > >> (AgentManager-Handler-8:null) (logid:) SeqA 3-438475:
> >> Processing
> >> > Seq
> >> > >> 3-438475: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags:
> 11,
> >> > >> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand"
> >> :{"_proxyVmId":1,
> >> > >> "_loadInfo":"{\n
> >> > >> \"connections\": []\n}","wait":0}}] }
> >> > >> 2017-01-18 12:17:14,857 DEBUG [c.c.a.m.AgentManagerImpl]
> >> > >> (AgentManager-Handler-8:null) (logid:) SeqA 3-438475:
> Sending
> >> Seq
> >> > >> 3-438475: { Ans: , MgmtId: 279278805450954, via: 3, Ver:
> v1,
> >> > Flags:
> >> > >> 100010,
> >> > >> [{"com.cloud.agent.api.AgentControlAnswer":{"result":
> >> true,"wait":0}}]
> >> > >> }
> >> > >> 2017-01-18 12:17:15,984 INFO
> [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> > >> (AsyncJobMgr-Heartbeat-1:ctx-5682fe3c) (logid:901efb52)
> Begin
> >> > cleanup
> >> > >> expired async-jobs
> >> > >> 2017-01-18 12:17:15,990 INFO
> [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> > >> (AsyncJobMgr-Heartbeat-1:ctx-5682fe3c) (logid:901efb52)
> End
> >> > cleanup
> >> > >> expired async-jobs
> >> > >> 2017-01-18 12:17:16,046 DEBUG
> >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> > >> (RouterStatusMonitor-1:ctx-7ad0b18a) (logid:b5b19f2b)
> Found 0
> >> > routers
> >> > >> to update status.
> >> > >> 2017-01-18 12:17:16,047 DEBUG
> >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> > >> (RouterStatusMonitor-1:ctx-7ad0b18a) (logid:b5b19f2b)
> Found 0
> >> VPC
> >> > >> networks to update Redundant State.
> >> > >> 2017-01-18 12:17:16,049 DEBUG
> >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> > >> (RouterStatusMonitor-1:ctx-7ad0b18a) (logid:b5b19f2b)
> Found 0
> >> > networks
> >> > >> to update RvR status.
> >> > >> 2017-01-18 12:17:16,097 DEBUG
> >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> > >> (RouterStatusMonitor-1:ctx-10a8aefd) (logid:b09ffa05)
> Found 0
> >> > routers
> >> > >> to update status.
> >> > >> 2017-01-18 12:17:16,099 DEBUG
> >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> > >> (RouterStatusMonitor-1:ctx-10a8aefd) (logid:b09ffa05)
> Found 0
> >> VPC
> >> > >> networks to update Redundant State.
> >> > >> 2017-01-18 12:17:16,101 DEBUG
> >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> > >> (RouterStatusMonitor-1:ctx-10a8aefd) (logid:b09ffa05)
> Found 0
> >> > networks
> >> > >> to update RvR status.
> >> > >> 2017-01-18 12:17:19,799 DEBUG [c.c.a.m.AgentManagerImpl]
> >> > >> (AgentManager-Handler-3:null) (logid:) SeqA 3-438476:
> >> Processing
> >> > Seq
> >> > >> 3-438476: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags:
> 11,
> >> > >> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand"
> >> :{"_proxyVmId":1,
> >> > >> "_loadInfo":"{\n
> >> > >> \"connections\": []\n}","wait":0}}] }
> >> > >> 2017-01-18 12:17:19,925 DEBUG [c.c.a.m.AgentManagerImpl]
> >> > >> (AgentManager-Handler-3:null) (logid:) SeqA 3-438476:
> Sending
> >> Seq
> >> > >> 3-438476: { Ans: , MgmtId: 279278805450954, via: 3, Ver:
> v1,
> >> > Flags:
> >> > >> 100010,
> >> > >> [{"com.cloud.agent.api.AgentControlAnswer":{"result":
> >> true,"wait":0}}]
> >> > >> }
> >> > >> 2017-01-18 12:17:25,985 INFO
> [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> > >> (AsyncJobMgr-Heartbeat-1:ctx-a4c8ceb4) (logid:d0d85ed6)
> Begin
> >> > cleanup
> >> > >> expired async-jobs
> >> > >> 2017-01-18 12:17:25,991 INFO
> [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> > >> (AsyncJobMgr-Heartbeat-1:ctx-a4c8ceb4) (logid:d0d85ed6)
> End
> >> > cleanup
> >> > >> expired async-jobs
> >> > >> 2017-01-18 12:17:29,800 DEBUG [c.c.a.m.AgentManagerImpl]
> >> > >> (AgentManager-Handler-7:null) (logid:) SeqA 3-438477:
> >> Processing
> >> > Seq
> >> > >> 3-438477: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags:
> 11,
> >> > >> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand"
> >> :{"_proxyVmId":1,
> >> > >> "_loadInfo":"{\n
> >> > >> \"connections\": []\n}","wait":0}}] }
> >> > >> 2017-01-18 12:17:29,860 DEBUG [c.c.a.m.AgentManagerImpl]
> >> > >> (AgentManager-Handler-7:null) (logid:) SeqA 3-438477:
> Sending
> >> Seq
> >> > >> 3-438477: { Ans: , MgmtId: 279278805450954, via: 3, Ver:
> v1,
> >> > Flags:
> >> > >> 100010,
> >> > >> [{"com.cloud.agent.api.AgentControlAnswer":{"result":
> >> true,"wait":0}}]
> >> > >> }
> >> > >> 2017-01-18 12:17:30,844 DEBUG
> >> > [o.a.c.s.SecondaryStorageManagerImpl]
> >> > >> (secstorage-1:ctx-f2d92c7a) (logid:6bcbea1f) Zone 1 is
> ready to
> >> > launch
> >> > >> secondary storage VM
> >> > >> 2017-01-18 12:17:31,095 DEBUG
> [c.c.c.ConsoleProxyManagerImpl]
> >> > >> (consoleproxy-1:ctx-5c5c1698) (logid:a199e87c) Zone 1 is
> ready
> >> to
> >> > >> launch console proxy
> >> > >> 2017-01-18 12:17:34,801 DEBUG [c.c.a.m.AgentManagerImpl]
> >> > >> (AgentManager-Handler-4:null) (logid:) SeqA 3-438478:
> >> Processing
> >> > Seq
> >> > >> 3-438478: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags:
> 11,
> >> > >> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand"
> >> :{"_proxyVmId":1,
> >> > >> "_loadInfo":"{\n
> >> > >> \"connections\": []\n}","wait":0}}] }
> >> > >> 2017-01-18 12:17:34,853 DEBUG [c.c.a.m.AgentManagerImpl]
> >> > >> (AgentManager-Handler-4:null) (logid:) SeqA 3-438478:
> Sending
> >> Seq
> >> > >> 3-438478: { Ans: , MgmtId: 279278805450954, via: 3, Ver:
> v1,
> >> > Flags:
> >> > >> 100010,
> >> > >> [{"com.cloud.agent.api.AgentControlAnswer":{"result":
> >> true,"wait":0}}]
> >> > >> }
> >> > >> 2017-01-18 12:17:35,534 DEBUG [c.c.s.StatsCollector]
> >> > >> (StatsCollector-4:ctx-37da523c) (logid:aecb957a)
> AutoScaling
> >> > Monitor
> >> > >> is running...
> >> > >> 2017-01-18 12:17:35,984 INFO
> [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> > >> (AsyncJobMgr-Heartbeat-1:ctx-d031d5da) (logid:722c8ae3)
> Begin
> >> > cleanup
> >> > >> expired async-jobs
> >> > >> 2017-01-18 12:17:35,990 INFO
> [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> > >> (AsyncJobMgr-Heartbeat-1:ctx-d031d5da) (logid:722c8ae3)
> End
> >> > cleanup
> >> > >> expired async-jobs
> >> > >> 2017-01-18 12:17:44,802 DEBUG [c.c.a.m.AgentManagerImpl]
> >> > >> (AgentManager-Handler-1:null) (logid:) SeqA 3-438479:
> >> Processing
> >> > Seq
> >> > >> 3-438479: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags:
> 11,
> >> > >> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand"
> >> :{"_proxyVmId":1,
> >> > >> "_loadInfo":"{\n
> >> > >> \"connections\": []\n}","wait":0}}] }
> >> > >> 2017-01-18 12:17:44,863 DEBUG [c.c.a.m.AgentManagerImpl]
> >> > >> (AgentManager-Handler-1:null) (logid:) SeqA 3-438479:
> Sending
> >> Seq
> >> > >> 3-438479: { Ans: , MgmtId: 279278805450954, via: 3, Ver:
> v1,
> >> > Flags:
> >> > >> 100010,
> >> > >> [{"com.cloud.agent.api.AgentControlAnswer":{"result":
> >> true,"wait":0}}]
> >> > >> }
> >> > >> 2017-01-18 12:17:45,754 DEBUG [c.c.a.m.AgentManagerImpl]
> >> > >> (AgentManager-Handler-12:null) (logid:) Ping from
> 1(iuatc)
> >> > >> 2017-01-18 12:17:45,754 DEBUG
> >> > [c.c.v.VirtualMachinePowerStateSyncImpl]
> >> > >> (AgentManager-Handler-12:null) (logid:) Process host VM
> state
> >> > report
> >> > >> from ping process. host: 1
> >> > >> 2017-01-18 12:17:45,760 DEBUG
> >> > [c.c.v.VirtualMachinePowerStateSyncImpl]
> >> > >> (AgentManager-Handler-12:null) (logid:) Process VM state
> >> report.
> >> > host:
> >> > >> 1, number of records in report: 2
> >> > >> 2017-01-18 12:17:45,760 DEBUG
> >> > [c.c.v.VirtualMachinePowerStateSyncImpl]
> >> > >> (AgentManager-Handler-12:null) (logid:) VM state report.
> host:
> >> 1,
> >> > vm
> >> > >> id:
> >> > >> 1, power state: PowerOn
> >> > >> 2017-01-18 12:17:45,763 DEBUG
> >> > [c.c.v.VirtualMachinePowerStateSyncImpl]
> >> > >> (AgentManager-Handler-12:null) (logid:) VM power state
> does not
> >> > >> change, skip DB writing. vm id: 1
> >> > >> 2017-01-18 12:17:45,763 DEBUG
> >> > [c.c.v.VirtualMachinePowerStateSyncImpl]
> >> > >> (AgentManager-Handler-12:null) (logid:) VM state report.
> host:
> >> 1,
> >> > vm
> >> > >> id:
> >> > >> 2, power state: PowerOn
> >> > >> 2017-01-18 12:17:45,766 DEBUG
> >> > [c.c.v.VirtualMachinePowerStateSyncImpl]
> >> > >> (AgentManager-Handler-12:null) (logid:) VM power state
> does not
> >> > >> change, skip DB writing. vm id: 2
> >> > >> 2017-01-18 12:17:45,769 DEBUG
> >> > [c.c.v.VirtualMachinePowerStateSyncImpl]
> >> > >> (AgentManager-Handler-12:null) (logid:) Done with process
> of VM
> >> > state
> >> > >> report. host: 1
> >> > >> 2017-01-18 12:17:45,985 INFO
> [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> > >> (AsyncJobMgr-Heartbeat-1:ctx-b39e7e84) (logid:d0f90660)
> Begin
> >> > cleanup
> >> > >> expired async-jobs
> >> > >> 2017-01-18 12:17:45,991 INFO
> [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> > >> (AsyncJobMgr-Heartbeat-1:ctx-b39e7e84) (logid:d0f90660)
> End
> >> > cleanup
> >> > >> expired async-jobs
> >> > >> 2017-01-18 12:17:46,046 DEBUG
> >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> > >> (RouterStatusMonitor-1:ctx-36fbb5c2) (logid:d346d004)
> Found 0
> >> > routers
> >> > >> to update status.
> >> > >> 2017-01-18 12:17:46,047 DEBUG
> >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> > >> (RouterStatusMonitor-1:ctx-36fbb5c2) (logid:d346d004)
> Found 0
> >> VPC
> >> > >> networks to update Redundant State.
> >> > >> 2017-01-18 12:17:46,049 DEBUG
> >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> > >> (RouterStatusMonitor-1:ctx-36fbb5c2) (logid:d346d004)
> Found 0
> >> > networks
> >> > >> to update RvR status.
> >> > >> 2017-01-18 12:17:46,097 DEBUG
> >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> > >> (RouterStatusMonitor-1:ctx-16234697) (logid:beb5fec7)
> Found 0
> >> > routers
> >> > >> to update status.
> >> > >> 2017-01-18 12:17:46,099 DEBUG
> >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> > >> (RouterStatusMonitor-1:ctx-16234697) (logid:beb5fec7)
> Found 0
> >> VPC
> >> > >> networks to update Redundant State.
> >> > >> 2017-01-18 12:17:46,100 DEBUG
> >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> > >> (RouterStatusMonitor-1:ctx-16234697) (logid:beb5fec7)
> Found 0
> >> > networks
> >> > >> to update RvR status.
> >> > >> 2017-01-18 12:17:46,201 DEBUG [c.c.a.m.AgentManagerImpl]
> >> > >> (AgentManager-Handler-2:null) (logid:) Ping from
> 2(s-2-VM)
> >> > >> 2017-01-18 12:17:47,521 DEBUG [c.c.a.m.AgentManagerImpl]
> >> > >> (AgentManager-Handler-14:null) (logid:) Ping from
> 3(v-1-VM)
> >> > >> 2017-01-18 12:17:49,842 DEBUG [c.c.a.m.AgentManagerImpl]
> >> > >> (AgentManager-Handler-13:null) (logid:) SeqA 3-438481:
> >> Processing
> >> > Seq
> >> > >> 3-438481: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags:
> 11,
> >> > >> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand"
> >> :{"_proxyVmId":1,
> >> > >> "_loadInfo":"{\n
> >> > >> \"connections\": []\n}","wait":0}}] }
> >> > >> 2017-01-18 12:17:49,897 DEBUG [c.c.a.m.AgentManagerImpl]
> >> > >> (AgentManager-Handler-13:null) (logid:) SeqA 3-438481:
> Sending
> >> Seq
> >> > >> 3-438481: { Ans: , MgmtId: 279278805450954, via: 3, Ver:
> v1,
> >> > Flags:
> >> > >> 100010,
> >> > >> [{"com.cloud.agent.api.AgentControlAnswer":{"result":
> >> true,"wait":0}}]
> >> > >> }
> >> > >> 2017-01-18 12:17:55,985 INFO
> [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> > >> (AsyncJobMgr-Heartbeat-1:ctx-c0db35b5) (logid:c304f172)
> Begin
> >> > cleanup
> >> > >> expired async-jobs
> >> > >> 2017-01-18 12:17:55,991 INFO
> [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> > >> (AsyncJobMgr-Heartbeat-1:ctx-c0db35b5) (logid:c304f172)
> End
> >> > cleanup
> >> > >> expired async-jobs
> >> > >> 2017-01-18 12:17:59,250 DEBUG [c.c.s.StatsCollector]
> >> > >> (StatsCollector-6:ctx-eec84206) (logid:b88e0ce8)
> >> VmStatsCollector
> >> > is
> >> > >> running...
> >> > >> 2017-01-18 12:17:59,803 DEBUG [c.c.a.m.AgentManagerImpl]
> >> > >> (AgentManager-Handler-11:null) (logid:) SeqA 3-438482:
> >> Processing
> >> > Seq
> >> > >> 3-438482: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags:
> 11,
> >> > >> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand"
> >> :{"_proxyVmId":1,
> >> > >> "_loadInfo":"{\n
> >> > >> \"connections\": []\n}","wait":0}}] }
> >> > >> 2017-01-18 12:17:59,857 DEBUG [c.c.a.m.AgentManagerImpl]
> >> > >> (AgentManager-Handler-11:null) (logid:) SeqA 3-438482:
> Sending
> >> Seq
> >> > >> 3-438482: { Ans: , MgmtId: 279278805450954, via: 3, Ver:
> v1,
> >> > Flags:
> >> > >> 100010,
> >> > >> [{"com.cloud.agent.api.AgentControlAnswer":{"result":
> >> true,"wait":0}}]
> >> > >> }
> >> > >> 2017-01-18 12:18:00,843 DEBUG
> >> > [o.a.c.s.SecondaryStorageManagerImpl]
> >> > >> (secstorage-1:ctx-f45be089) (logid:934bd9f7) Zone 1 is
> ready to
> >> > launch
> >> > >> secondary storage VM
> >> > >> 2017-01-18 12:18:01,094 DEBUG
> [c.c.c.ConsoleProxyManagerImpl]
> >> > >> (consoleproxy-1:ctx-4b11dfe0) (logid:cce52dbb) Zone 1 is
> ready
> >> to
> >> > >> launch console proxy
> >> > >> 2017-01-18 12:18:03,345 DEBUG [c.c.s.StatsCollector]
> >> > >> (StatsCollector-2:ctx-c743d4ee) (logid:c1f62e31)
> >> HostStatsCollector
> >> > is
> >> > >> running...
> >> > >> 2017-01-18 12:18:03,406 DEBUG [c.c.a.t.Request]
> >> > >> (StatsCollector-2:ctx-c743d4ee) (logid:c1f62e31) Seq
> >> > >> 1-6227070909769882805: Received: { Ans: , MgmtId:
> >> > 279278805450954,
> >> > >> via:
> >> > >> 1(iuatc), Ver: v1, Flags: 10, { GetHostStatsAnswer } }
> >> > >> 2017-01-18 12:18:04,804 DEBUG [c.c.a.m.AgentManagerImpl]
> >> > >> (AgentManager-Handler-9:null) (logid:) SeqA 3-438483:
> >> Processing
> >> > Seq
> >> > >> 3-438483: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags:
> 11,
> >> > >> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand"
> >> :{"_proxyVmId":1,
> >> > >> "_loadInfo":"{\n
> >> > >> \"connections\": []\n}","wait":0}}] }
> >> > >> 2017-01-18 12:18:04,858 DEBUG [c.c.a.m.AgentManagerImpl]
> >> > >> (AgentManager-Handler-9:null) (logid:) SeqA 3-438483:
> Sending
> >> Seq
> >> > >> 3-438483: { Ans: , MgmtId: 279278805450954, via: 3, Ver:
> v1,
> >> > Flags:
> >> > >> 100010,
> >> > >> [{"com.cloud.agent.api.AgentControlAnswer":{"result":
> >> true,"wait":0}}]
> >> > >> }
> >> > >> 2017-01-18 12:18:05,985 INFO
> [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> > >> (AsyncJobMgr-Heartbeat-1:ctx-149380f2) (logid:53d166f9)
> Begin
> >> > cleanup
> >> > >> expired async-jobs
> >> > >> 2017-01-18 12:18:05,990 INFO
> [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> > >> (AsyncJobMgr-Heartbeat-1:ctx-149380f2) (logid:53d166f9)
> End
> >> > cleanup
> >> > >> expired async-jobs
> >> > >> 2017-01-18 12:18:14,805 DEBUG [c.c.a.m.AgentManagerImpl]
> >> > >> (AgentManager-Handler-5:null) (logid:) SeqA 3-438484:
> >> Processing
> >> > Seq
> >> > >> 3-438484: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags:
> 11,
> >> > >> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand"
> >> :{"_proxyVmId":1,
> >> > >> "_loadInfo":"{\n
> >> > >> \"connections\": []\n}","wait":0}}] }
> >> > >> 2017-01-18 12:18:14,951 DEBUG [c.c.a.m.AgentManagerImpl]
> >> > >> (AgentManager-Handler-5:null) (logid:) SeqA 3-438484:
> Sending
> >> Seq
> >> > >> 3-438484: { Ans: , MgmtId: 279278805450954, via: 3, Ver:
> v1,
> >> > Flags:
> >> > >> 100010,
> >> > >> [{"com.cloud.agent.api.AgentControlAnswer":{"result":
> >> true,"wait":0}}]
> >> > >> }
> >> > >> 2017-01-18 12:18:15,985 INFO
> [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> > >> (AsyncJobMgr-Heartbeat-1:ctx-7ccabf83) (logid:2b0f34c4)
> Begin
> >> > cleanup
> >> > >> expired async-jobs
> >> > >> 2017-01-18 12:18:15,993 INFO
> [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> > >> (AsyncJobMgr-Heartbeat-1:ctx-7ccabf83) (logid:2b0f34c4)
> End
> >> > cleanup
> >> > >> expired async-jobs
> >> > >> 2017-01-18 12:18:16,046 DEBUG
> >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> > >> (RouterStatusMonitor-1:ctx-fc5408a0) (logid:bbd625c1)
> Found 0
> >> > routers
> >> > >> to update status.
> >> > >> 2017-01-18 12:18:16,048 DEBUG
> >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> > >> (RouterStatusMonitor-1:ctx-fc5408a0) (logid:bbd625c1)
> Found 0
> >> VPC
> >> > >> networks to update Redundant State.
> >> > >> 2017-01-18 12:18:16,049 DEBUG
> >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> > >> (RouterStatusMonitor-1:ctx-fc5408a0) (logid:bbd625c1)
> Found 0
> >> > networks
> >> > >> to update RvR status.
> >> > >> 2017-01-18 12:18:16,097 DEBUG
> >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> > >> (RouterStatusMonitor-1:ctx-2c697f6b) (logid:cfa955f2)
> Found 0
> >> > routers
> >> > >> to update status.
> >> > >> 2017-01-18 12:18:16,099 DEBUG
> >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> > >> (RouterStatusMonitor-1:ctx-2c697f6b) (logid:cfa955f2)
> Found 0
> >> VPC
> >> > >> networks to update Redundant State.
> >> > >> 2017-01-18 12:18:16,101 DEBUG
> >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> > >> (RouterStatusMonitor-1:ctx-2c697f6b) (logid:cfa955f2)
> Found 0
> >> > networks
> >> > >> to update RvR status.
> >> > >> 2017-01-18 12:18:24,389 DEBUG [c.c.h.d.HostDaoImpl]
> >> > >> (ClusteredAgentManager
> >> > >> Timer:ctx-d699e7ed) (logid:e10673e8) Resetting hosts
> suitable
> >> for
> >> > >> reconnect
> >> > >> 2017-01-18 12:18:24,392 DEBUG [c.c.h.d.HostDaoImpl]
> >> > >> (ClusteredAgentManager
> >> > >> Timer:ctx-d699e7ed) (logid:e10673e8) Completed resetting
> hosts
> >> > >> suitable for reconnect
> >> > >> 2017-01-18 12:18:24,392 DEBUG [c.c.h.d.HostDaoImpl]
> >> > >> (ClusteredAgentManager
> >> > >> Timer:ctx-d699e7ed) (logid:e10673e8) Acquiring hosts for
> >> clusters
> >> > >> already owned by this management server
> >> > >> 2017-01-18 12:18:24,392 DEBUG [c.c.h.d.HostDaoImpl]
> >> > >> (ClusteredAgentManager
> >> > >> Timer:ctx-d699e7ed) (logid:e10673e8) Completed acquiring
> hosts
> >> for
> >> > >> clusters already owned by this management server
> >> > >> 2017-01-18 12:18:24,392 DEBUG [c.c.h.d.HostDaoImpl]
> >> > >> (ClusteredAgentManager
> >> > >> Timer:ctx-d699e7ed) (logid:e10673e8) Acquiring hosts for
> >> clusters
> >> > not
> >> > >> owned by any management server
> >> > >> 2017-01-18 12:18:24,393 DEBUG [c.c.h.d.HostDaoImpl]
> >> > >> (ClusteredAgentManager
> >> > >> Timer:ctx-d699e7ed) (logid:e10673e8) Completed acquiring
> hosts
> >> for
> >> > >> clusters not owned by any management server
> >> > >> 2017-01-18 12:18:24,807 DEBUG [c.c.a.m.AgentManagerImpl]
> >> > >> (AgentManager-Handler-6:null) (logid:) SeqA 3-438485:
> >> Processing
> >> > Seq
> >> > >> 3-438485: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags:
> 11,
> >> > >> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand"
> >> :{"_proxyVmId":1,
> >> > >> "_loadInfo":"{\n
> >> > >> \"connections\": []\n}","wait":0}}] }
> >> > >> 2017-01-18 12:18:24,870 DEBUG [c.c.a.m.AgentManagerImpl]
> >> > >> (AgentManager-Handler-6:null) (logid:) SeqA 3-438485:
> Sending
> >> Seq
> >> > >> 3-438485: { Ans: , MgmtId: 279278805450954, via: 3, Ver:
> v1,
> >> > Flags:
> >> > >> 100010,
> >> [{"com.cloud.agent.api.AgentControlAnswer":{"result":tru
> >> > >>
> >> > >>> Can you please share your management_server logs
> >> > >>>
> >> > >>> On Tue, Jan 17, 2017 at 11:47 PM,
> >> > Ã\u0192¬Ã\u201a¡Ã\u201a°Ã\u0192«Ã\u2026â\u20ac\u2122ââ\u20ac\u0161¬Ã\u0192ÂÃ\u2039Å\u201cââ\u201a¬Â¢
> >> > >>> <ca...@renet.kr> wrote:
> >> > >>>
> >> > >>>> Hi, all
> >> > >>>>
> >> > >>>> I am facing secondary storage problem.
> >> > >>>>
> >> > >>>> I am using NFS Server to use primary and secondary
> storage.
> >> > >>>>
> >> > >>>> After rebooting the host,
> >> > >>>>
> >> > >>>> In Dashboard Cloudstack , it says Secondary Storage 0.00
> KB /
> >> > 0.00
> >> > >>>> KB
> >> > >>>>
> >> > >>>> Primary Storage is okay.
> >> > >>>>
> >> > >>>>
> >> > >>>>
> >> > >>>> After all, system VMs are not up. It was working fine
> before
> >> > >>>> rebooting.
> >> > >>>>
> >> > >>>>
> >> > >>>>
> >> > >>>> I have tried several times of restart of Management
> service.
> >> > >>>>
> >> > >>>>
> >> > >>>>
> >> > >>>> Any ideas?
> >> > >>>>
> >> > >>>>
> >> > >>>>
> >> > >>>> Thanks,
> >> > >>>>
> >> > >>>>
> >> > >>>
> >> > >>
> >> > >>
> >> > >>
> >> > >
> >> > >
> >> > >
> >> >
> >> >
> >> >
> >> >
> >> >
> >> > Dag.Sonstebo@shapeblue.com
> >> > www.shapeblue.com
> >> > 53 Chandos Place, Covent Garden, London WC2N 4HSUK
> >> > @shapeblue
> >> >
> >> >
> >> >
> >> >
> >>
> >>
> >>
> >
>
>
>
>
>
> Dag.Sonstebo@shapeblue.com
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London WC2N 4HSUK
> @shapeblue
>
>
>
>
Re: Cloudstack 4.7 Secondary Storage problem
Posted by Dag Sonstebo <Da...@shapeblue.com>.
Merlin,
Can you actually ping 10.8.19.111? Have you set up the NFS shares according to CloudStack specification ( /export <CIDR>(rw,async,no_root_squash,no_subtree_check) )? What is the contents of your /etc/exports file?
In short it looks like you have either a connectivity or permissions issue – which is unrelated to CloudStack. Until you have this sorted you won’t be able to spin up system VMs etc. With regards to ACLs on your NFS share – keep in mind you need to open this up to any IP address in your management IP range – since the SSVM can appear on any IP in this.
Regards,
Dag Sonstebo
Cloud Architect
ShapeBlue
On 18/01/2017, 12:10, "merlin@students.iitmandi.ac.in" <me...@students.iitmandi.ac.in> wrote:
It did not throw me any error this time, however, there was no response
either. The cursor just kept blinking for a very long time. untill I did
ctrl+c.
Also, a brief observation: when I write mount in my nfs server:
I see three lines at the end
10.8.19.111:/export/primary on /mnt/38454858-08bb-3e92-88c9-354465ad3715
type nfs
(rw,noac,actimeo=0,vers=4,addr=10.8.19.111,clientaddr=10.8.19.111)
10.8.19.111:/export/primary on /mnt/primary type nfs
(rw,rsize=8192,wsize=8192,timeo=14,intr,vers=3,noac,actimeo=0,addr=10.8.19.111)
10.8.19.111:/export/secondary on /mnt/secondary type nfs
(rw,rsize=8192,wsize=8192,timeo=14,intr,vers=3,noac,actimeo=0,addr=10.8.19.111)
Which used to be just 2 when I was using cloudstack version 4.4. I
recently installed 4.8 and I see this after clean installation.
> try "mount -t nfs 10.8.19.111:/export/secondary/ /tmp/secondary"
>
> *Vivek Kumar*
> Virtualization and Cloud Consultant
>
> [image: http://www.indiqus.com/images/logo.jpg] <http://www.indiqus.com/>
> *I*ndi*Q*us Technologies Pvt Ltd
> A-98, LGF, C.R.Park, New Delhi - 110019
> *O* +91 11 4055 1411 | *M* +91 7503460090
> www.indiqus.com <http://www.indiqus.com/>
>
> On Wed, Jan 18, 2017 at 5:23 PM, <me...@students.iitmandi.ac.in> wrote:
>
>> Hi,
>>
>> I tried like you said: here is the result
>>
>> root@s-2-VM:~# mount -t NFS 10.8.19.111:/export/secondary/
>> /tmp/secondary/
>> mount: unknown filesystem type 'NFS'
>> root@s-2-VM:~# mount -t 10.8.19.111:/export/secondary/ /tmp/secondary/
>> nfs
>> mount: mount point nfs does not exist
>>
>> I might be running it wrong. Also, i checked the ssvm for the nfs
>> package
>> and neither was it there, nor could i download.
>> Please help
>>
>>
>> > Hi Merlin,
>> >
>> > What Paul means is you need to log on to the SSVM, then do something
>> like
>> > this:
>> >
>> > mkdir /tmp/secondary
>> > mount –t NFS <NFS server IP>:/<NFS/path/here>/ /tmp/secondary
>> >
>> > This will tell you if your NFS permissions are configured correctly.
>> >
>> > Regards,
>> > Dag Sonstebo
>> > Cloud Architect
>> > ShapeBlue
>> >
>> > On 18/01/2017, 09:29, "merlin@students.iitmandi.ac.in"
>> > <me...@students.iitmandi.ac.in> wrote:
>> >
>> > Also when you say mount from ssvm. Does it mean this?
>> > Log into ssvm and run "mount":- the reply is down below. I am not
>> able
>> > to
>> > understand much even from the documentation,
>> > I am sorry for the lack of full knowledge in this concern.
>> >
>> >
>> > sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
>> > proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
>> > udev on /dev type devtmpfs
>> > (rw,relatime,size=10240k,nr_inodes=62128,mode=755)
>> > devpts on /dev/pts type devpts
>> > (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
>> > tmpfs on /run type tmpfs
>> > (rw,nosuid,noexec,relatime,size=50896k,mode=755)
>> > /dev/disk/by-uuid/30c81d3d-ee9f-4a88-81c1-5f349b22ba1d on / type
>> ext4
>> > (rw,relatime,errors=remount-ro,user_xattr,barrier=1,data=ordered)
>> > tmpfs on /run/lock type tmpfs
>> > (rw,nosuid,nodev,noexec,relatime,size=5120k)
>> > tmpfs on /run/shm type tmpfs
>> > (rw,nosuid,nodev,noexec,relatime,size=211960k)
>> > /dev/vda1 on /boot type ext4
>> > (rw,relatime,user_xattr,barrier=1,data=ordered)
>> > /dev/vda6 on /home type ext4
>> > (rw,relatime,user_xattr,barrier=1,data=ordered)
>> > /dev/vda8 on /opt type ext4
>> > (rw,relatime,user_xattr,barrier=1,data=ordered)
>> > /dev/vda11 on /tmp type ext4
>> > (rw,relatime,user_xattr,barrier=1,data=ordered)
>> > /dev/vda7 on /usr type ext4
>> > (rw,relatime,user_xattr,barrier=1,data=ordered)
>> > /dev/vda9 on /var type ext4
>> > (rw,relatime,user_xattr,barrier=1,data=ordered)
>> > /dev/vda10 on /var/log type ext4
>> > (rw,relatime,user_xattr,barrier=1,data=ordered)
>> > rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs
>> (rw,relatime)
>> >
>> > > Hi Merlin,
>> > >
>> > > To troubleshoot, try mounting the NFS yourself from the SSVM and
>> get
>> > that
>> > > working.
>> > >
>> > > the likely problem is permissions on the NFS mounts. Either the
>> IP
>> > of the
>> > > SSVM is not allowed or the permissions don't include
>> no_root_squash
>> > (or
>> > > equivalent) the SSVM is probably connecting as 'cloud', so
>> > connections
>> > > have to be mapped to 'root'. Finally the SSVM only reliably
>> > supports
>> > > NFSv3
>> > >
>> > >
>> > >
>> > >
>> > > Kind regards,
>> > >
>> > > Paul Angus
>> > >
>> > > paul.angus@shapeblue.comÂ
>> > > www.shapeblue.com
>> > > 53 Chandos Place, Covent Garden, London WC2N 4HSUK
>> > > @shapeblue
>> > >
>> > >
>> > >
>> > >
>> > > -----Original Message-----
>> > > From: merlin@students.iitmandi.ac.in
>> > > [mailto:merlin@students.iitmandi.ac.in]
>> > > Sent: 18 January 2017 08:48
>> > > To: users@cloudstack.apache.org
>> > > Subject: Re: Cloudstack 4.7 Secondary Storage problem
>> > >
>> > > And this is the result of check.sh
>> > >
>> > > 2 packets transmitted, 2 packets received, 0% packet loss
>> > round-trip
>> > > min/avg/max/stddev = 0.400/0.550/0.701/0.151 ms
>> > > Good: Can ping DNS server
>> > > ================================================
>> > > Good: DNS resolves download.cloud.com
>> > > ================================================
>> > > ERROR: NFS is not currently mounted
>> > > Try manually mounting from inside the VM NFS server is 10.8.0.1
>> > PING
>> > > 10.8.0.1 (10.8.0.1): 48 data bytes
>> > > 56 bytes from 10.8.0.1: icmp_seq=0 ttl=64 time=0.456 ms
>> > > 56 bytes from 10.8.0.1: icmp_seq=1 ttl=64 time=0.708 ms
>> > > --- 10.8.0.1 ping statistics ---
>> > > 2 packets transmitted, 2 packets received, 0% packet loss
>> > round-trip
>> > > min/avg/max/stddev = 0.456/0.582/0.708/0.126 ms
>> > > Good: Can ping nfs server
>> > > ================================================
>> > > Management server is 10.8.19.111. Checking connectivity.
>> > > Good: Can connect to management server port 8250
>> > > ================================================
>> > > Good: Java process is running
>> > > ================================================
>> > > Tests Complete. Look for ERROR or WARNING above.
>> > >
>> > >
>> > > The problem is that while installation I did mount nfs?
>> > > I dont understand what else to do next
>> > >
>> > >
>> > >
>> > >> Hi, even I am facing the same problem.. and here are the logs
>> from
>> > >> today:
>> > >>
>> > >>
>> > >>
>> > >> 2017-01-18 12:16:46,100 DEBUG
>> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> > >> (RouterStatusMonitor-1:ctx-bc5cb4cd) (logid:0ffac658) Found 0
>> > networks
>> > >> to update RvR status.
>> > >> 2017-01-18 12:16:46,200 DEBUG [c.c.a.m.AgentManagerImpl]
>> > >> (AgentManager-Handler-11:null) (logid:) Ping from 2(s-2-VM)
>> > >> 2017-01-18 12:16:47,521 DEBUG [c.c.a.m.AgentManagerImpl]
>> > >> (AgentManager-Handler-15:null) (logid:) Ping from 3(v-1-VM)
>> > >> 2017-01-18 12:16:49,834 DEBUG [c.c.a.m.AgentManagerImpl]
>> > >> (AgentManager-Handler-9:null) (logid:) SeqA 3-438472:
>> Processing
>> > Seq
>> > >> 3-438472: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
>> > >> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand"
>> :{"_proxyVmId":1,
>> > >> "_loadInfo":"{\n
>> > >> \"connections\": []\n}","wait":0}}] }
>> > >> 2017-01-18 12:16:49,887 DEBUG [c.c.a.m.AgentManagerImpl]
>> > >> (AgentManager-Handler-9:null) (logid:) SeqA 3-438472: Sending
>> Seq
>> > >> 3-438472: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1,
>> > Flags:
>> > >> 100010,
>> > >> [{"com.cloud.agent.api.AgentControlAnswer":{"result":
>> true,"wait":0}}]
>> > >> }
>> > >> 2017-01-18 12:16:54,390 DEBUG [c.c.h.d.HostDaoImpl]
>> > >> (ClusteredAgentManager
>> > >> Timer:ctx-8464e4b3) (logid:1f15c721) Resetting hosts suitable
>> for
>> > >> reconnect
>> > >> 2017-01-18 12:16:54,392 DEBUG [c.c.h.d.HostDaoImpl]
>> > >> (ClusteredAgentManager
>> > >> Timer:ctx-8464e4b3) (logid:1f15c721) Completed resetting hosts
>> > >> suitable for reconnect
>> > >> 2017-01-18 12:16:54,392 DEBUG [c.c.h.d.HostDaoImpl]
>> > >> (ClusteredAgentManager
>> > >> Timer:ctx-8464e4b3) (logid:1f15c721) Acquiring hosts for
>> clusters
>> > >> already owned by this management server
>> > >> 2017-01-18 12:16:54,393 DEBUG [c.c.h.d.HostDaoImpl]
>> > >> (ClusteredAgentManager
>> > >> Timer:ctx-8464e4b3) (logid:1f15c721) Completed acquiring hosts
>> for
>> > >> clusters already owned by this management server
>> > >> 2017-01-18 12:16:54,393 DEBUG [c.c.h.d.HostDaoImpl]
>> > >> (ClusteredAgentManager
>> > >> Timer:ctx-8464e4b3) (logid:1f15c721) Acquiring hosts for
>> clusters
>> > not
>> > >> owned by any management server
>> > >> 2017-01-18 12:16:54,394 DEBUG [c.c.h.d.HostDaoImpl]
>> > >> (ClusteredAgentManager
>> > >> Timer:ctx-8464e4b3) (logid:1f15c721) Completed acquiring hosts
>> for
>> > >> clusters not owned by any management server
>> > >> 2017-01-18 12:16:54,795 DEBUG [c.c.a.m.AgentManagerImpl]
>> > >> (AgentManager-Handler-5:null) (logid:) SeqA 3-438473:
>> Processing
>> > Seq
>> > >> 3-438473: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
>> > >> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand"
>> :{"_proxyVmId":1,
>> > >> "_loadInfo":"{\n
>> > >> \"connections\": []\n}","wait":0}}] }
>> > >> 2017-01-18 12:16:54,854 DEBUG [c.c.a.m.AgentManagerImpl]
>> > >> (AgentManager-Handler-5:null) (logid:) SeqA 3-438473: Sending
>> Seq
>> > >> 3-438473: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1,
>> > Flags:
>> > >> 100010,
>> > >> [{"com.cloud.agent.api.AgentControlAnswer":{"result":
>> true,"wait":0}}]
>> > >> }
>> > >> 2017-01-18 12:16:55,984 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > >> (AsyncJobMgr-Heartbeat-1:ctx-d0469453) (logid:af5d4f47) Begin
>> > cleanup
>> > >> expired async-jobs
>> > >> 2017-01-18 12:16:55,991 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > >> (AsyncJobMgr-Heartbeat-1:ctx-d0469453) (logid:af5d4f47) End
>> > cleanup
>> > >> expired async-jobs
>> > >> 2017-01-18 12:16:59,245 DEBUG [c.c.s.StatsCollector]
>> > >> (StatsCollector-1:ctx-68069dff) (logid:a7a6974d)
>> VmStatsCollector
>> > is
>> > >> running...
>> > >> 2017-01-18 12:17:00,841 DEBUG
>> > [o.a.c.s.SecondaryStorageManagerImpl]
>> > >> (secstorage-1:ctx-4e101aa5) (logid:65f1f0cb) Zone 1 is ready to
>> > launch
>> > >> secondary storage VM
>> > >> 2017-01-18 12:17:01,095 DEBUG [c.c.c.ConsoleProxyManagerImpl]
>> > >> (consoleproxy-1:ctx-393828fa) (logid:7b3d0491) Zone 1 is ready
>> to
>> > >> launch console proxy
>> > >> 2017-01-18 12:17:03,283 DEBUG [c.c.s.StatsCollector]
>> > >> (StatsCollector-3:ctx-cc1f58a1) (logid:3ae41340)
>> HostStatsCollector
>> > is
>> > >> running...
>> > >> 2017-01-18 12:17:03,343 DEBUG [c.c.a.t.Request]
>> > >> (StatsCollector-3:ctx-cc1f58a1) (logid:3ae41340) Seq
>> > >> 1-6227070909769882804: Received: { Ans: , MgmtId:
>> > 279278805450954,
>> > >> via:
>> > >> 1(iuatc), Ver: v1, Flags: 10, { GetHostStatsAnswer } }
>> > >> 2017-01-18 12:17:04,796 DEBUG [c.c.a.m.AgentManagerImpl]
>> > >> (AgentManager-Handler-10:null) (logid:) SeqA 3-438474:
>> Processing
>> > Seq
>> > >> 3-438474: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
>> > >> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand"
>> :{"_proxyVmId":1,
>> > >> "_loadInfo":"{\n
>> > >> \"connections\": []\n}","wait":0}}] }
>> > >> 2017-01-18 12:17:04,847 DEBUG [c.c.a.m.AgentManagerImpl]
>> > >> (AgentManager-Handler-10:null) (logid:) SeqA 3-438474: Sending
>> Seq
>> > >> 3-438474: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1,
>> > Flags:
>> > >> 100010,
>> > >> [{"com.cloud.agent.api.AgentControlAnswer":{"result":
>> true,"wait":0}}]
>> > >> }
>> > >> 2017-01-18 12:17:05,985 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > >> (AsyncJobMgr-Heartbeat-1:ctx-7f0de3b8) (logid:e9bbd63a) Begin
>> > cleanup
>> > >> expired async-jobs
>> > >> 2017-01-18 12:17:05,991 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > >> (AsyncJobMgr-Heartbeat-1:ctx-7f0de3b8) (logid:e9bbd63a) End
>> > cleanup
>> > >> expired async-jobs
>> > >> 2017-01-18 12:17:14,798 DEBUG [c.c.a.m.AgentManagerImpl]
>> > >> (AgentManager-Handler-8:null) (logid:) SeqA 3-438475:
>> Processing
>> > Seq
>> > >> 3-438475: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
>> > >> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand"
>> :{"_proxyVmId":1,
>> > >> "_loadInfo":"{\n
>> > >> \"connections\": []\n}","wait":0}}] }
>> > >> 2017-01-18 12:17:14,857 DEBUG [c.c.a.m.AgentManagerImpl]
>> > >> (AgentManager-Handler-8:null) (logid:) SeqA 3-438475: Sending
>> Seq
>> > >> 3-438475: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1,
>> > Flags:
>> > >> 100010,
>> > >> [{"com.cloud.agent.api.AgentControlAnswer":{"result":
>> true,"wait":0}}]
>> > >> }
>> > >> 2017-01-18 12:17:15,984 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > >> (AsyncJobMgr-Heartbeat-1:ctx-5682fe3c) (logid:901efb52) Begin
>> > cleanup
>> > >> expired async-jobs
>> > >> 2017-01-18 12:17:15,990 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > >> (AsyncJobMgr-Heartbeat-1:ctx-5682fe3c) (logid:901efb52) End
>> > cleanup
>> > >> expired async-jobs
>> > >> 2017-01-18 12:17:16,046 DEBUG
>> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> > >> (RouterStatusMonitor-1:ctx-7ad0b18a) (logid:b5b19f2b) Found 0
>> > routers
>> > >> to update status.
>> > >> 2017-01-18 12:17:16,047 DEBUG
>> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> > >> (RouterStatusMonitor-1:ctx-7ad0b18a) (logid:b5b19f2b) Found 0
>> VPC
>> > >> networks to update Redundant State.
>> > >> 2017-01-18 12:17:16,049 DEBUG
>> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> > >> (RouterStatusMonitor-1:ctx-7ad0b18a) (logid:b5b19f2b) Found 0
>> > networks
>> > >> to update RvR status.
>> > >> 2017-01-18 12:17:16,097 DEBUG
>> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> > >> (RouterStatusMonitor-1:ctx-10a8aefd) (logid:b09ffa05) Found 0
>> > routers
>> > >> to update status.
>> > >> 2017-01-18 12:17:16,099 DEBUG
>> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> > >> (RouterStatusMonitor-1:ctx-10a8aefd) (logid:b09ffa05) Found 0
>> VPC
>> > >> networks to update Redundant State.
>> > >> 2017-01-18 12:17:16,101 DEBUG
>> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> > >> (RouterStatusMonitor-1:ctx-10a8aefd) (logid:b09ffa05) Found 0
>> > networks
>> > >> to update RvR status.
>> > >> 2017-01-18 12:17:19,799 DEBUG [c.c.a.m.AgentManagerImpl]
>> > >> (AgentManager-Handler-3:null) (logid:) SeqA 3-438476:
>> Processing
>> > Seq
>> > >> 3-438476: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
>> > >> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand"
>> :{"_proxyVmId":1,
>> > >> "_loadInfo":"{\n
>> > >> \"connections\": []\n}","wait":0}}] }
>> > >> 2017-01-18 12:17:19,925 DEBUG [c.c.a.m.AgentManagerImpl]
>> > >> (AgentManager-Handler-3:null) (logid:) SeqA 3-438476: Sending
>> Seq
>> > >> 3-438476: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1,
>> > Flags:
>> > >> 100010,
>> > >> [{"com.cloud.agent.api.AgentControlAnswer":{"result":
>> true,"wait":0}}]
>> > >> }
>> > >> 2017-01-18 12:17:25,985 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > >> (AsyncJobMgr-Heartbeat-1:ctx-a4c8ceb4) (logid:d0d85ed6) Begin
>> > cleanup
>> > >> expired async-jobs
>> > >> 2017-01-18 12:17:25,991 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > >> (AsyncJobMgr-Heartbeat-1:ctx-a4c8ceb4) (logid:d0d85ed6) End
>> > cleanup
>> > >> expired async-jobs
>> > >> 2017-01-18 12:17:29,800 DEBUG [c.c.a.m.AgentManagerImpl]
>> > >> (AgentManager-Handler-7:null) (logid:) SeqA 3-438477:
>> Processing
>> > Seq
>> > >> 3-438477: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
>> > >> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand"
>> :{"_proxyVmId":1,
>> > >> "_loadInfo":"{\n
>> > >> \"connections\": []\n}","wait":0}}] }
>> > >> 2017-01-18 12:17:29,860 DEBUG [c.c.a.m.AgentManagerImpl]
>> > >> (AgentManager-Handler-7:null) (logid:) SeqA 3-438477: Sending
>> Seq
>> > >> 3-438477: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1,
>> > Flags:
>> > >> 100010,
>> > >> [{"com.cloud.agent.api.AgentControlAnswer":{"result":
>> true,"wait":0}}]
>> > >> }
>> > >> 2017-01-18 12:17:30,844 DEBUG
>> > [o.a.c.s.SecondaryStorageManagerImpl]
>> > >> (secstorage-1:ctx-f2d92c7a) (logid:6bcbea1f) Zone 1 is ready to
>> > launch
>> > >> secondary storage VM
>> > >> 2017-01-18 12:17:31,095 DEBUG [c.c.c.ConsoleProxyManagerImpl]
>> > >> (consoleproxy-1:ctx-5c5c1698) (logid:a199e87c) Zone 1 is ready
>> to
>> > >> launch console proxy
>> > >> 2017-01-18 12:17:34,801 DEBUG [c.c.a.m.AgentManagerImpl]
>> > >> (AgentManager-Handler-4:null) (logid:) SeqA 3-438478:
>> Processing
>> > Seq
>> > >> 3-438478: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
>> > >> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand"
>> :{"_proxyVmId":1,
>> > >> "_loadInfo":"{\n
>> > >> \"connections\": []\n}","wait":0}}] }
>> > >> 2017-01-18 12:17:34,853 DEBUG [c.c.a.m.AgentManagerImpl]
>> > >> (AgentManager-Handler-4:null) (logid:) SeqA 3-438478: Sending
>> Seq
>> > >> 3-438478: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1,
>> > Flags:
>> > >> 100010,
>> > >> [{"com.cloud.agent.api.AgentControlAnswer":{"result":
>> true,"wait":0}}]
>> > >> }
>> > >> 2017-01-18 12:17:35,534 DEBUG [c.c.s.StatsCollector]
>> > >> (StatsCollector-4:ctx-37da523c) (logid:aecb957a) AutoScaling
>> > Monitor
>> > >> is running...
>> > >> 2017-01-18 12:17:35,984 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > >> (AsyncJobMgr-Heartbeat-1:ctx-d031d5da) (logid:722c8ae3) Begin
>> > cleanup
>> > >> expired async-jobs
>> > >> 2017-01-18 12:17:35,990 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > >> (AsyncJobMgr-Heartbeat-1:ctx-d031d5da) (logid:722c8ae3) End
>> > cleanup
>> > >> expired async-jobs
>> > >> 2017-01-18 12:17:44,802 DEBUG [c.c.a.m.AgentManagerImpl]
>> > >> (AgentManager-Handler-1:null) (logid:) SeqA 3-438479:
>> Processing
>> > Seq
>> > >> 3-438479: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
>> > >> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand"
>> :{"_proxyVmId":1,
>> > >> "_loadInfo":"{\n
>> > >> \"connections\": []\n}","wait":0}}] }
>> > >> 2017-01-18 12:17:44,863 DEBUG [c.c.a.m.AgentManagerImpl]
>> > >> (AgentManager-Handler-1:null) (logid:) SeqA 3-438479: Sending
>> Seq
>> > >> 3-438479: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1,
>> > Flags:
>> > >> 100010,
>> > >> [{"com.cloud.agent.api.AgentControlAnswer":{"result":
>> true,"wait":0}}]
>> > >> }
>> > >> 2017-01-18 12:17:45,754 DEBUG [c.c.a.m.AgentManagerImpl]
>> > >> (AgentManager-Handler-12:null) (logid:) Ping from 1(iuatc)
>> > >> 2017-01-18 12:17:45,754 DEBUG
>> > [c.c.v.VirtualMachinePowerStateSyncImpl]
>> > >> (AgentManager-Handler-12:null) (logid:) Process host VM state
>> > report
>> > >> from ping process. host: 1
>> > >> 2017-01-18 12:17:45,760 DEBUG
>> > [c.c.v.VirtualMachinePowerStateSyncImpl]
>> > >> (AgentManager-Handler-12:null) (logid:) Process VM state
>> report.
>> > host:
>> > >> 1, number of records in report: 2
>> > >> 2017-01-18 12:17:45,760 DEBUG
>> > [c.c.v.VirtualMachinePowerStateSyncImpl]
>> > >> (AgentManager-Handler-12:null) (logid:) VM state report. host:
>> 1,
>> > vm
>> > >> id:
>> > >> 1, power state: PowerOn
>> > >> 2017-01-18 12:17:45,763 DEBUG
>> > [c.c.v.VirtualMachinePowerStateSyncImpl]
>> > >> (AgentManager-Handler-12:null) (logid:) VM power state does not
>> > >> change, skip DB writing. vm id: 1
>> > >> 2017-01-18 12:17:45,763 DEBUG
>> > [c.c.v.VirtualMachinePowerStateSyncImpl]
>> > >> (AgentManager-Handler-12:null) (logid:) VM state report. host:
>> 1,
>> > vm
>> > >> id:
>> > >> 2, power state: PowerOn
>> > >> 2017-01-18 12:17:45,766 DEBUG
>> > [c.c.v.VirtualMachinePowerStateSyncImpl]
>> > >> (AgentManager-Handler-12:null) (logid:) VM power state does not
>> > >> change, skip DB writing. vm id: 2
>> > >> 2017-01-18 12:17:45,769 DEBUG
>> > [c.c.v.VirtualMachinePowerStateSyncImpl]
>> > >> (AgentManager-Handler-12:null) (logid:) Done with process of VM
>> > state
>> > >> report. host: 1
>> > >> 2017-01-18 12:17:45,985 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > >> (AsyncJobMgr-Heartbeat-1:ctx-b39e7e84) (logid:d0f90660) Begin
>> > cleanup
>> > >> expired async-jobs
>> > >> 2017-01-18 12:17:45,991 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > >> (AsyncJobMgr-Heartbeat-1:ctx-b39e7e84) (logid:d0f90660) End
>> > cleanup
>> > >> expired async-jobs
>> > >> 2017-01-18 12:17:46,046 DEBUG
>> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> > >> (RouterStatusMonitor-1:ctx-36fbb5c2) (logid:d346d004) Found 0
>> > routers
>> > >> to update status.
>> > >> 2017-01-18 12:17:46,047 DEBUG
>> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> > >> (RouterStatusMonitor-1:ctx-36fbb5c2) (logid:d346d004) Found 0
>> VPC
>> > >> networks to update Redundant State.
>> > >> 2017-01-18 12:17:46,049 DEBUG
>> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> > >> (RouterStatusMonitor-1:ctx-36fbb5c2) (logid:d346d004) Found 0
>> > networks
>> > >> to update RvR status.
>> > >> 2017-01-18 12:17:46,097 DEBUG
>> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> > >> (RouterStatusMonitor-1:ctx-16234697) (logid:beb5fec7) Found 0
>> > routers
>> > >> to update status.
>> > >> 2017-01-18 12:17:46,099 DEBUG
>> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> > >> (RouterStatusMonitor-1:ctx-16234697) (logid:beb5fec7) Found 0
>> VPC
>> > >> networks to update Redundant State.
>> > >> 2017-01-18 12:17:46,100 DEBUG
>> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> > >> (RouterStatusMonitor-1:ctx-16234697) (logid:beb5fec7) Found 0
>> > networks
>> > >> to update RvR status.
>> > >> 2017-01-18 12:17:46,201 DEBUG [c.c.a.m.AgentManagerImpl]
>> > >> (AgentManager-Handler-2:null) (logid:) Ping from 2(s-2-VM)
>> > >> 2017-01-18 12:17:47,521 DEBUG [c.c.a.m.AgentManagerImpl]
>> > >> (AgentManager-Handler-14:null) (logid:) Ping from 3(v-1-VM)
>> > >> 2017-01-18 12:17:49,842 DEBUG [c.c.a.m.AgentManagerImpl]
>> > >> (AgentManager-Handler-13:null) (logid:) SeqA 3-438481:
>> Processing
>> > Seq
>> > >> 3-438481: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
>> > >> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand"
>> :{"_proxyVmId":1,
>> > >> "_loadInfo":"{\n
>> > >> \"connections\": []\n}","wait":0}}] }
>> > >> 2017-01-18 12:17:49,897 DEBUG [c.c.a.m.AgentManagerImpl]
>> > >> (AgentManager-Handler-13:null) (logid:) SeqA 3-438481: Sending
>> Seq
>> > >> 3-438481: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1,
>> > Flags:
>> > >> 100010,
>> > >> [{"com.cloud.agent.api.AgentControlAnswer":{"result":
>> true,"wait":0}}]
>> > >> }
>> > >> 2017-01-18 12:17:55,985 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > >> (AsyncJobMgr-Heartbeat-1:ctx-c0db35b5) (logid:c304f172) Begin
>> > cleanup
>> > >> expired async-jobs
>> > >> 2017-01-18 12:17:55,991 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > >> (AsyncJobMgr-Heartbeat-1:ctx-c0db35b5) (logid:c304f172) End
>> > cleanup
>> > >> expired async-jobs
>> > >> 2017-01-18 12:17:59,250 DEBUG [c.c.s.StatsCollector]
>> > >> (StatsCollector-6:ctx-eec84206) (logid:b88e0ce8)
>> VmStatsCollector
>> > is
>> > >> running...
>> > >> 2017-01-18 12:17:59,803 DEBUG [c.c.a.m.AgentManagerImpl]
>> > >> (AgentManager-Handler-11:null) (logid:) SeqA 3-438482:
>> Processing
>> > Seq
>> > >> 3-438482: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
>> > >> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand"
>> :{"_proxyVmId":1,
>> > >> "_loadInfo":"{\n
>> > >> \"connections\": []\n}","wait":0}}] }
>> > >> 2017-01-18 12:17:59,857 DEBUG [c.c.a.m.AgentManagerImpl]
>> > >> (AgentManager-Handler-11:null) (logid:) SeqA 3-438482: Sending
>> Seq
>> > >> 3-438482: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1,
>> > Flags:
>> > >> 100010,
>> > >> [{"com.cloud.agent.api.AgentControlAnswer":{"result":
>> true,"wait":0}}]
>> > >> }
>> > >> 2017-01-18 12:18:00,843 DEBUG
>> > [o.a.c.s.SecondaryStorageManagerImpl]
>> > >> (secstorage-1:ctx-f45be089) (logid:934bd9f7) Zone 1 is ready to
>> > launch
>> > >> secondary storage VM
>> > >> 2017-01-18 12:18:01,094 DEBUG [c.c.c.ConsoleProxyManagerImpl]
>> > >> (consoleproxy-1:ctx-4b11dfe0) (logid:cce52dbb) Zone 1 is ready
>> to
>> > >> launch console proxy
>> > >> 2017-01-18 12:18:03,345 DEBUG [c.c.s.StatsCollector]
>> > >> (StatsCollector-2:ctx-c743d4ee) (logid:c1f62e31)
>> HostStatsCollector
>> > is
>> > >> running...
>> > >> 2017-01-18 12:18:03,406 DEBUG [c.c.a.t.Request]
>> > >> (StatsCollector-2:ctx-c743d4ee) (logid:c1f62e31) Seq
>> > >> 1-6227070909769882805: Received: { Ans: , MgmtId:
>> > 279278805450954,
>> > >> via:
>> > >> 1(iuatc), Ver: v1, Flags: 10, { GetHostStatsAnswer } }
>> > >> 2017-01-18 12:18:04,804 DEBUG [c.c.a.m.AgentManagerImpl]
>> > >> (AgentManager-Handler-9:null) (logid:) SeqA 3-438483:
>> Processing
>> > Seq
>> > >> 3-438483: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
>> > >> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand"
>> :{"_proxyVmId":1,
>> > >> "_loadInfo":"{\n
>> > >> \"connections\": []\n}","wait":0}}] }
>> > >> 2017-01-18 12:18:04,858 DEBUG [c.c.a.m.AgentManagerImpl]
>> > >> (AgentManager-Handler-9:null) (logid:) SeqA 3-438483: Sending
>> Seq
>> > >> 3-438483: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1,
>> > Flags:
>> > >> 100010,
>> > >> [{"com.cloud.agent.api.AgentControlAnswer":{"result":
>> true,"wait":0}}]
>> > >> }
>> > >> 2017-01-18 12:18:05,985 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > >> (AsyncJobMgr-Heartbeat-1:ctx-149380f2) (logid:53d166f9) Begin
>> > cleanup
>> > >> expired async-jobs
>> > >> 2017-01-18 12:18:05,990 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > >> (AsyncJobMgr-Heartbeat-1:ctx-149380f2) (logid:53d166f9) End
>> > cleanup
>> > >> expired async-jobs
>> > >> 2017-01-18 12:18:14,805 DEBUG [c.c.a.m.AgentManagerImpl]
>> > >> (AgentManager-Handler-5:null) (logid:) SeqA 3-438484:
>> Processing
>> > Seq
>> > >> 3-438484: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
>> > >> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand"
>> :{"_proxyVmId":1,
>> > >> "_loadInfo":"{\n
>> > >> \"connections\": []\n}","wait":0}}] }
>> > >> 2017-01-18 12:18:14,951 DEBUG [c.c.a.m.AgentManagerImpl]
>> > >> (AgentManager-Handler-5:null) (logid:) SeqA 3-438484: Sending
>> Seq
>> > >> 3-438484: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1,
>> > Flags:
>> > >> 100010,
>> > >> [{"com.cloud.agent.api.AgentControlAnswer":{"result":
>> true,"wait":0}}]
>> > >> }
>> > >> 2017-01-18 12:18:15,985 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > >> (AsyncJobMgr-Heartbeat-1:ctx-7ccabf83) (logid:2b0f34c4) Begin
>> > cleanup
>> > >> expired async-jobs
>> > >> 2017-01-18 12:18:15,993 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > >> (AsyncJobMgr-Heartbeat-1:ctx-7ccabf83) (logid:2b0f34c4) End
>> > cleanup
>> > >> expired async-jobs
>> > >> 2017-01-18 12:18:16,046 DEBUG
>> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> > >> (RouterStatusMonitor-1:ctx-fc5408a0) (logid:bbd625c1) Found 0
>> > routers
>> > >> to update status.
>> > >> 2017-01-18 12:18:16,048 DEBUG
>> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> > >> (RouterStatusMonitor-1:ctx-fc5408a0) (logid:bbd625c1) Found 0
>> VPC
>> > >> networks to update Redundant State.
>> > >> 2017-01-18 12:18:16,049 DEBUG
>> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> > >> (RouterStatusMonitor-1:ctx-fc5408a0) (logid:bbd625c1) Found 0
>> > networks
>> > >> to update RvR status.
>> > >> 2017-01-18 12:18:16,097 DEBUG
>> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> > >> (RouterStatusMonitor-1:ctx-2c697f6b) (logid:cfa955f2) Found 0
>> > routers
>> > >> to update status.
>> > >> 2017-01-18 12:18:16,099 DEBUG
>> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> > >> (RouterStatusMonitor-1:ctx-2c697f6b) (logid:cfa955f2) Found 0
>> VPC
>> > >> networks to update Redundant State.
>> > >> 2017-01-18 12:18:16,101 DEBUG
>> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> > >> (RouterStatusMonitor-1:ctx-2c697f6b) (logid:cfa955f2) Found 0
>> > networks
>> > >> to update RvR status.
>> > >> 2017-01-18 12:18:24,389 DEBUG [c.c.h.d.HostDaoImpl]
>> > >> (ClusteredAgentManager
>> > >> Timer:ctx-d699e7ed) (logid:e10673e8) Resetting hosts suitable
>> for
>> > >> reconnect
>> > >> 2017-01-18 12:18:24,392 DEBUG [c.c.h.d.HostDaoImpl]
>> > >> (ClusteredAgentManager
>> > >> Timer:ctx-d699e7ed) (logid:e10673e8) Completed resetting hosts
>> > >> suitable for reconnect
>> > >> 2017-01-18 12:18:24,392 DEBUG [c.c.h.d.HostDaoImpl]
>> > >> (ClusteredAgentManager
>> > >> Timer:ctx-d699e7ed) (logid:e10673e8) Acquiring hosts for
>> clusters
>> > >> already owned by this management server
>> > >> 2017-01-18 12:18:24,392 DEBUG [c.c.h.d.HostDaoImpl]
>> > >> (ClusteredAgentManager
>> > >> Timer:ctx-d699e7ed) (logid:e10673e8) Completed acquiring hosts
>> for
>> > >> clusters already owned by this management server
>> > >> 2017-01-18 12:18:24,392 DEBUG [c.c.h.d.HostDaoImpl]
>> > >> (ClusteredAgentManager
>> > >> Timer:ctx-d699e7ed) (logid:e10673e8) Acquiring hosts for
>> clusters
>> > not
>> > >> owned by any management server
>> > >> 2017-01-18 12:18:24,393 DEBUG [c.c.h.d.HostDaoImpl]
>> > >> (ClusteredAgentManager
>> > >> Timer:ctx-d699e7ed) (logid:e10673e8) Completed acquiring hosts
>> for
>> > >> clusters not owned by any management server
>> > >> 2017-01-18 12:18:24,807 DEBUG [c.c.a.m.AgentManagerImpl]
>> > >> (AgentManager-Handler-6:null) (logid:) SeqA 3-438485:
>> Processing
>> > Seq
>> > >> 3-438485: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
>> > >> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand"
>> :{"_proxyVmId":1,
>> > >> "_loadInfo":"{\n
>> > >> \"connections\": []\n}","wait":0}}] }
>> > >> 2017-01-18 12:18:24,870 DEBUG [c.c.a.m.AgentManagerImpl]
>> > >> (AgentManager-Handler-6:null) (logid:) SeqA 3-438485: Sending
>> Seq
>> > >> 3-438485: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1,
>> > Flags:
>> > >> 100010,
>> [{"com.cloud.agent.api.AgentControlAnswer":{"result":tru
>> > >>
>> > >>> Can you please share your management_server logs
>> > >>>
>> > >>> On Tue, Jan 17, 2017 at 11:47 PM,
>> > 조대Ø•
>> > >>> <ca...@renet.kr> wrote:
>> > >>>
>> > >>>> Hi, all
>> > >>>>
>> > >>>> I am facing secondary storage problem.
>> > >>>>
>> > >>>> I am using NFS Server to use primary and secondary storage.
>> > >>>>
>> > >>>> After rebooting the host,
>> > >>>>
>> > >>>> In Dashboard Cloudstack , it says Secondary Storage 0.00 KB /
>> > 0.00
>> > >>>> KB
>> > >>>>
>> > >>>> Primary Storage is okay.
>> > >>>>
>> > >>>>
>> > >>>>
>> > >>>> After all, system VMs are not up. It was working fine before
>> > >>>> rebooting.
>> > >>>>
>> > >>>>
>> > >>>>
>> > >>>> I have tried several times of restart of Management service.
>> > >>>>
>> > >>>>
>> > >>>>
>> > >>>> Any ideas?
>> > >>>>
>> > >>>>
>> > >>>>
>> > >>>> Thanks,
>> > >>>>
>> > >>>>
>> > >>>
>> > >>
>> > >>
>> > >>
>> > >
>> > >
>> > >
>> >
>> >
>> >
>> >
>> >
>> > Dag.Sonstebo@shapeblue.com
>> > www.shapeblue.com
>> > 53 Chandos Place, Covent Garden, London WC2N 4HSUK
>> > @shapeblue
>> >
>> >
>> >
>> >
>>
>>
>>
>
Dag.Sonstebo@shapeblue.com
www.shapeblue.com
53 Chandos Place, Covent Garden, London WC2N 4HSUK
@shapeblue
Re: Cloudstack 4.7 Secondary Storage problem
Posted by me...@students.iitmandi.ac.in.
It did not throw me any error this time, however, there was no response
either. The cursor just kept blinking for a very long time. untill I did
ctrl+c.
Also, a brief observation: when I write mount in my nfs server:
I see three lines at the end
10.8.19.111:/export/primary on /mnt/38454858-08bb-3e92-88c9-354465ad3715
type nfs
(rw,noac,actimeo=0,vers=4,addr=10.8.19.111,clientaddr=10.8.19.111)
10.8.19.111:/export/primary on /mnt/primary type nfs
(rw,rsize=8192,wsize=8192,timeo=14,intr,vers=3,noac,actimeo=0,addr=10.8.19.111)
10.8.19.111:/export/secondary on /mnt/secondary type nfs
(rw,rsize=8192,wsize=8192,timeo=14,intr,vers=3,noac,actimeo=0,addr=10.8.19.111)
Which used to be just 2 when I was using cloudstack version 4.4. I
recently installed 4.8 and I see this after clean installation.
> try "mount -t nfs 10.8.19.111:/export/secondary/ /tmp/secondary"
>
> *Vivek Kumar*
> Virtualization and Cloud Consultant
>
> [image: http://www.indiqus.com/images/logo.jpg] <http://www.indiqus.com/>
> *I*ndi*Q*us Technologies Pvt Ltd
> A-98, LGF, C.R.Park, New Delhi - 110019
> *O* +91 11 4055 1411 | *M* +91 7503460090
> www.indiqus.com <http://www.indiqus.com/>
>
> On Wed, Jan 18, 2017 at 5:23 PM, <me...@students.iitmandi.ac.in> wrote:
>
>> Hi,
>>
>> I tried like you said: here is the result
>>
>> root@s-2-VM:~# mount -t NFS 10.8.19.111:/export/secondary/
>> /tmp/secondary/
>> mount: unknown filesystem type 'NFS'
>> root@s-2-VM:~# mount -t 10.8.19.111:/export/secondary/ /tmp/secondary/
>> nfs
>> mount: mount point nfs does not exist
>>
>> I might be running it wrong. Also, i checked the ssvm for the nfs
>> package
>> and neither was it there, nor could i download.
>> Please help
>>
>>
>> > Hi Merlin,
>> >
>> > What Paul means is you need to log on to the SSVM, then do something
>> like
>> > this:
>> >
>> > mkdir /tmp/secondary
>> > mount \u2013t NFS <NFS server IP>:/<NFS/path/here>/ /tmp/secondary
>> >
>> > This will tell you if your NFS permissions are configured correctly.
>> >
>> > Regards,
>> > Dag Sonstebo
>> > Cloud Architect
>> > ShapeBlue
>> >
>> > On 18/01/2017, 09:29, "merlin@students.iitmandi.ac.in"
>> > <me...@students.iitmandi.ac.in> wrote:
>> >
>> > Also when you say mount from ssvm. Does it mean this?
>> > Log into ssvm and run "mount":- the reply is down below. I am not
>> able
>> > to
>> > understand much even from the documentation,
>> > I am sorry for the lack of full knowledge in this concern.
>> >
>> >
>> > sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
>> > proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
>> > udev on /dev type devtmpfs
>> > (rw,relatime,size=10240k,nr_inodes=62128,mode=755)
>> > devpts on /dev/pts type devpts
>> > (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
>> > tmpfs on /run type tmpfs
>> > (rw,nosuid,noexec,relatime,size=50896k,mode=755)
>> > /dev/disk/by-uuid/30c81d3d-ee9f-4a88-81c1-5f349b22ba1d on / type
>> ext4
>> > (rw,relatime,errors=remount-ro,user_xattr,barrier=1,data=ordered)
>> > tmpfs on /run/lock type tmpfs
>> > (rw,nosuid,nodev,noexec,relatime,size=5120k)
>> > tmpfs on /run/shm type tmpfs
>> > (rw,nosuid,nodev,noexec,relatime,size=211960k)
>> > /dev/vda1 on /boot type ext4
>> > (rw,relatime,user_xattr,barrier=1,data=ordered)
>> > /dev/vda6 on /home type ext4
>> > (rw,relatime,user_xattr,barrier=1,data=ordered)
>> > /dev/vda8 on /opt type ext4
>> > (rw,relatime,user_xattr,barrier=1,data=ordered)
>> > /dev/vda11 on /tmp type ext4
>> > (rw,relatime,user_xattr,barrier=1,data=ordered)
>> > /dev/vda7 on /usr type ext4
>> > (rw,relatime,user_xattr,barrier=1,data=ordered)
>> > /dev/vda9 on /var type ext4
>> > (rw,relatime,user_xattr,barrier=1,data=ordered)
>> > /dev/vda10 on /var/log type ext4
>> > (rw,relatime,user_xattr,barrier=1,data=ordered)
>> > rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs
>> (rw,relatime)
>> >
>> > > Hi Merlin,
>> > >
>> > > To troubleshoot, try mounting the NFS yourself from the SSVM and
>> get
>> > that
>> > > working.
>> > >
>> > > the likely problem is permissions on the NFS mounts. Either the
>> IP
>> > of the
>> > > SSVM is not allowed or the permissions don't include
>> no_root_squash
>> > (or
>> > > equivalent) the SSVM is probably connecting as 'cloud', so
>> > connections
>> > > have to be mapped to 'root'. Finally the SSVM only reliably
>> > supports
>> > > NFSv3
>> > >
>> > >
>> > >
>> > >
>> > > Kind regards,
>> > >
>> > > Paul Angus
>> > >
>> > > paul.angus@shapeblue.comÂ
>> > > www.shapeblue.com
>> > > 53 Chandos Place, Covent Garden, London WC2N 4HSUK
>> > > @shapeblue
>> > >
>> > >
>> > >
>> > >
>> > > -----Original Message-----
>> > > From: merlin@students.iitmandi.ac.in
>> > > [mailto:merlin@students.iitmandi.ac.in]
>> > > Sent: 18 January 2017 08:48
>> > > To: users@cloudstack.apache.org
>> > > Subject: Re: Cloudstack 4.7 Secondary Storage problem
>> > >
>> > > And this is the result of check.sh
>> > >
>> > > 2 packets transmitted, 2 packets received, 0% packet loss
>> > round-trip
>> > > min/avg/max/stddev = 0.400/0.550/0.701/0.151 ms
>> > > Good: Can ping DNS server
>> > > ================================================
>> > > Good: DNS resolves download.cloud.com
>> > > ================================================
>> > > ERROR: NFS is not currently mounted
>> > > Try manually mounting from inside the VM NFS server is 10.8.0.1
>> > PING
>> > > 10.8.0.1 (10.8.0.1): 48 data bytes
>> > > 56 bytes from 10.8.0.1: icmp_seq=0 ttl=64 time=0.456 ms
>> > > 56 bytes from 10.8.0.1: icmp_seq=1 ttl=64 time=0.708 ms
>> > > --- 10.8.0.1 ping statistics ---
>> > > 2 packets transmitted, 2 packets received, 0% packet loss
>> > round-trip
>> > > min/avg/max/stddev = 0.456/0.582/0.708/0.126 ms
>> > > Good: Can ping nfs server
>> > > ================================================
>> > > Management server is 10.8.19.111. Checking connectivity.
>> > > Good: Can connect to management server port 8250
>> > > ================================================
>> > > Good: Java process is running
>> > > ================================================
>> > > Tests Complete. Look for ERROR or WARNING above.
>> > >
>> > >
>> > > The problem is that while installation I did mount nfs?
>> > > I dont understand what else to do next
>> > >
>> > >
>> > >
>> > >> Hi, even I am facing the same problem.. and here are the logs
>> from
>> > >> today:
>> > >>
>> > >>
>> > >>
>> > >> 2017-01-18 12:16:46,100 DEBUG
>> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> > >> (RouterStatusMonitor-1:ctx-bc5cb4cd) (logid:0ffac658) Found 0
>> > networks
>> > >> to update RvR status.
>> > >> 2017-01-18 12:16:46,200 DEBUG [c.c.a.m.AgentManagerImpl]
>> > >> (AgentManager-Handler-11:null) (logid:) Ping from 2(s-2-VM)
>> > >> 2017-01-18 12:16:47,521 DEBUG [c.c.a.m.AgentManagerImpl]
>> > >> (AgentManager-Handler-15:null) (logid:) Ping from 3(v-1-VM)
>> > >> 2017-01-18 12:16:49,834 DEBUG [c.c.a.m.AgentManagerImpl]
>> > >> (AgentManager-Handler-9:null) (logid:) SeqA 3-438472:
>> Processing
>> > Seq
>> > >> 3-438472: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
>> > >> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand"
>> :{"_proxyVmId":1,
>> > >> "_loadInfo":"{\n
>> > >> \"connections\": []\n}","wait":0}}] }
>> > >> 2017-01-18 12:16:49,887 DEBUG [c.c.a.m.AgentManagerImpl]
>> > >> (AgentManager-Handler-9:null) (logid:) SeqA 3-438472: Sending
>> Seq
>> > >> 3-438472: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1,
>> > Flags:
>> > >> 100010,
>> > >> [{"com.cloud.agent.api.AgentControlAnswer":{"result":
>> true,"wait":0}}]
>> > >> }
>> > >> 2017-01-18 12:16:54,390 DEBUG [c.c.h.d.HostDaoImpl]
>> > >> (ClusteredAgentManager
>> > >> Timer:ctx-8464e4b3) (logid:1f15c721) Resetting hosts suitable
>> for
>> > >> reconnect
>> > >> 2017-01-18 12:16:54,392 DEBUG [c.c.h.d.HostDaoImpl]
>> > >> (ClusteredAgentManager
>> > >> Timer:ctx-8464e4b3) (logid:1f15c721) Completed resetting hosts
>> > >> suitable for reconnect
>> > >> 2017-01-18 12:16:54,392 DEBUG [c.c.h.d.HostDaoImpl]
>> > >> (ClusteredAgentManager
>> > >> Timer:ctx-8464e4b3) (logid:1f15c721) Acquiring hosts for
>> clusters
>> > >> already owned by this management server
>> > >> 2017-01-18 12:16:54,393 DEBUG [c.c.h.d.HostDaoImpl]
>> > >> (ClusteredAgentManager
>> > >> Timer:ctx-8464e4b3) (logid:1f15c721) Completed acquiring hosts
>> for
>> > >> clusters already owned by this management server
>> > >> 2017-01-18 12:16:54,393 DEBUG [c.c.h.d.HostDaoImpl]
>> > >> (ClusteredAgentManager
>> > >> Timer:ctx-8464e4b3) (logid:1f15c721) Acquiring hosts for
>> clusters
>> > not
>> > >> owned by any management server
>> > >> 2017-01-18 12:16:54,394 DEBUG [c.c.h.d.HostDaoImpl]
>> > >> (ClusteredAgentManager
>> > >> Timer:ctx-8464e4b3) (logid:1f15c721) Completed acquiring hosts
>> for
>> > >> clusters not owned by any management server
>> > >> 2017-01-18 12:16:54,795 DEBUG [c.c.a.m.AgentManagerImpl]
>> > >> (AgentManager-Handler-5:null) (logid:) SeqA 3-438473:
>> Processing
>> > Seq
>> > >> 3-438473: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
>> > >> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand"
>> :{"_proxyVmId":1,
>> > >> "_loadInfo":"{\n
>> > >> \"connections\": []\n}","wait":0}}] }
>> > >> 2017-01-18 12:16:54,854 DEBUG [c.c.a.m.AgentManagerImpl]
>> > >> (AgentManager-Handler-5:null) (logid:) SeqA 3-438473: Sending
>> Seq
>> > >> 3-438473: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1,
>> > Flags:
>> > >> 100010,
>> > >> [{"com.cloud.agent.api.AgentControlAnswer":{"result":
>> true,"wait":0}}]
>> > >> }
>> > >> 2017-01-18 12:16:55,984 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > >> (AsyncJobMgr-Heartbeat-1:ctx-d0469453) (logid:af5d4f47) Begin
>> > cleanup
>> > >> expired async-jobs
>> > >> 2017-01-18 12:16:55,991 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > >> (AsyncJobMgr-Heartbeat-1:ctx-d0469453) (logid:af5d4f47) End
>> > cleanup
>> > >> expired async-jobs
>> > >> 2017-01-18 12:16:59,245 DEBUG [c.c.s.StatsCollector]
>> > >> (StatsCollector-1:ctx-68069dff) (logid:a7a6974d)
>> VmStatsCollector
>> > is
>> > >> running...
>> > >> 2017-01-18 12:17:00,841 DEBUG
>> > [o.a.c.s.SecondaryStorageManagerImpl]
>> > >> (secstorage-1:ctx-4e101aa5) (logid:65f1f0cb) Zone 1 is ready to
>> > launch
>> > >> secondary storage VM
>> > >> 2017-01-18 12:17:01,095 DEBUG [c.c.c.ConsoleProxyManagerImpl]
>> > >> (consoleproxy-1:ctx-393828fa) (logid:7b3d0491) Zone 1 is ready
>> to
>> > >> launch console proxy
>> > >> 2017-01-18 12:17:03,283 DEBUG [c.c.s.StatsCollector]
>> > >> (StatsCollector-3:ctx-cc1f58a1) (logid:3ae41340)
>> HostStatsCollector
>> > is
>> > >> running...
>> > >> 2017-01-18 12:17:03,343 DEBUG [c.c.a.t.Request]
>> > >> (StatsCollector-3:ctx-cc1f58a1) (logid:3ae41340) Seq
>> > >> 1-6227070909769882804: Received: { Ans: , MgmtId:
>> > 279278805450954,
>> > >> via:
>> > >> 1(iuatc), Ver: v1, Flags: 10, { GetHostStatsAnswer } }
>> > >> 2017-01-18 12:17:04,796 DEBUG [c.c.a.m.AgentManagerImpl]
>> > >> (AgentManager-Handler-10:null) (logid:) SeqA 3-438474:
>> Processing
>> > Seq
>> > >> 3-438474: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
>> > >> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand"
>> :{"_proxyVmId":1,
>> > >> "_loadInfo":"{\n
>> > >> \"connections\": []\n}","wait":0}}] }
>> > >> 2017-01-18 12:17:04,847 DEBUG [c.c.a.m.AgentManagerImpl]
>> > >> (AgentManager-Handler-10:null) (logid:) SeqA 3-438474: Sending
>> Seq
>> > >> 3-438474: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1,
>> > Flags:
>> > >> 100010,
>> > >> [{"com.cloud.agent.api.AgentControlAnswer":{"result":
>> true,"wait":0}}]
>> > >> }
>> > >> 2017-01-18 12:17:05,985 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > >> (AsyncJobMgr-Heartbeat-1:ctx-7f0de3b8) (logid:e9bbd63a) Begin
>> > cleanup
>> > >> expired async-jobs
>> > >> 2017-01-18 12:17:05,991 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > >> (AsyncJobMgr-Heartbeat-1:ctx-7f0de3b8) (logid:e9bbd63a) End
>> > cleanup
>> > >> expired async-jobs
>> > >> 2017-01-18 12:17:14,798 DEBUG [c.c.a.m.AgentManagerImpl]
>> > >> (AgentManager-Handler-8:null) (logid:) SeqA 3-438475:
>> Processing
>> > Seq
>> > >> 3-438475: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
>> > >> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand"
>> :{"_proxyVmId":1,
>> > >> "_loadInfo":"{\n
>> > >> \"connections\": []\n}","wait":0}}] }
>> > >> 2017-01-18 12:17:14,857 DEBUG [c.c.a.m.AgentManagerImpl]
>> > >> (AgentManager-Handler-8:null) (logid:) SeqA 3-438475: Sending
>> Seq
>> > >> 3-438475: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1,
>> > Flags:
>> > >> 100010,
>> > >> [{"com.cloud.agent.api.AgentControlAnswer":{"result":
>> true,"wait":0}}]
>> > >> }
>> > >> 2017-01-18 12:17:15,984 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > >> (AsyncJobMgr-Heartbeat-1:ctx-5682fe3c) (logid:901efb52) Begin
>> > cleanup
>> > >> expired async-jobs
>> > >> 2017-01-18 12:17:15,990 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > >> (AsyncJobMgr-Heartbeat-1:ctx-5682fe3c) (logid:901efb52) End
>> > cleanup
>> > >> expired async-jobs
>> > >> 2017-01-18 12:17:16,046 DEBUG
>> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> > >> (RouterStatusMonitor-1:ctx-7ad0b18a) (logid:b5b19f2b) Found 0
>> > routers
>> > >> to update status.
>> > >> 2017-01-18 12:17:16,047 DEBUG
>> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> > >> (RouterStatusMonitor-1:ctx-7ad0b18a) (logid:b5b19f2b) Found 0
>> VPC
>> > >> networks to update Redundant State.
>> > >> 2017-01-18 12:17:16,049 DEBUG
>> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> > >> (RouterStatusMonitor-1:ctx-7ad0b18a) (logid:b5b19f2b) Found 0
>> > networks
>> > >> to update RvR status.
>> > >> 2017-01-18 12:17:16,097 DEBUG
>> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> > >> (RouterStatusMonitor-1:ctx-10a8aefd) (logid:b09ffa05) Found 0
>> > routers
>> > >> to update status.
>> > >> 2017-01-18 12:17:16,099 DEBUG
>> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> > >> (RouterStatusMonitor-1:ctx-10a8aefd) (logid:b09ffa05) Found 0
>> VPC
>> > >> networks to update Redundant State.
>> > >> 2017-01-18 12:17:16,101 DEBUG
>> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> > >> (RouterStatusMonitor-1:ctx-10a8aefd) (logid:b09ffa05) Found 0
>> > networks
>> > >> to update RvR status.
>> > >> 2017-01-18 12:17:19,799 DEBUG [c.c.a.m.AgentManagerImpl]
>> > >> (AgentManager-Handler-3:null) (logid:) SeqA 3-438476:
>> Processing
>> > Seq
>> > >> 3-438476: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
>> > >> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand"
>> :{"_proxyVmId":1,
>> > >> "_loadInfo":"{\n
>> > >> \"connections\": []\n}","wait":0}}] }
>> > >> 2017-01-18 12:17:19,925 DEBUG [c.c.a.m.AgentManagerImpl]
>> > >> (AgentManager-Handler-3:null) (logid:) SeqA 3-438476: Sending
>> Seq
>> > >> 3-438476: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1,
>> > Flags:
>> > >> 100010,
>> > >> [{"com.cloud.agent.api.AgentControlAnswer":{"result":
>> true,"wait":0}}]
>> > >> }
>> > >> 2017-01-18 12:17:25,985 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > >> (AsyncJobMgr-Heartbeat-1:ctx-a4c8ceb4) (logid:d0d85ed6) Begin
>> > cleanup
>> > >> expired async-jobs
>> > >> 2017-01-18 12:17:25,991 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > >> (AsyncJobMgr-Heartbeat-1:ctx-a4c8ceb4) (logid:d0d85ed6) End
>> > cleanup
>> > >> expired async-jobs
>> > >> 2017-01-18 12:17:29,800 DEBUG [c.c.a.m.AgentManagerImpl]
>> > >> (AgentManager-Handler-7:null) (logid:) SeqA 3-438477:
>> Processing
>> > Seq
>> > >> 3-438477: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
>> > >> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand"
>> :{"_proxyVmId":1,
>> > >> "_loadInfo":"{\n
>> > >> \"connections\": []\n}","wait":0}}] }
>> > >> 2017-01-18 12:17:29,860 DEBUG [c.c.a.m.AgentManagerImpl]
>> > >> (AgentManager-Handler-7:null) (logid:) SeqA 3-438477: Sending
>> Seq
>> > >> 3-438477: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1,
>> > Flags:
>> > >> 100010,
>> > >> [{"com.cloud.agent.api.AgentControlAnswer":{"result":
>> true,"wait":0}}]
>> > >> }
>> > >> 2017-01-18 12:17:30,844 DEBUG
>> > [o.a.c.s.SecondaryStorageManagerImpl]
>> > >> (secstorage-1:ctx-f2d92c7a) (logid:6bcbea1f) Zone 1 is ready to
>> > launch
>> > >> secondary storage VM
>> > >> 2017-01-18 12:17:31,095 DEBUG [c.c.c.ConsoleProxyManagerImpl]
>> > >> (consoleproxy-1:ctx-5c5c1698) (logid:a199e87c) Zone 1 is ready
>> to
>> > >> launch console proxy
>> > >> 2017-01-18 12:17:34,801 DEBUG [c.c.a.m.AgentManagerImpl]
>> > >> (AgentManager-Handler-4:null) (logid:) SeqA 3-438478:
>> Processing
>> > Seq
>> > >> 3-438478: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
>> > >> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand"
>> :{"_proxyVmId":1,
>> > >> "_loadInfo":"{\n
>> > >> \"connections\": []\n}","wait":0}}] }
>> > >> 2017-01-18 12:17:34,853 DEBUG [c.c.a.m.AgentManagerImpl]
>> > >> (AgentManager-Handler-4:null) (logid:) SeqA 3-438478: Sending
>> Seq
>> > >> 3-438478: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1,
>> > Flags:
>> > >> 100010,
>> > >> [{"com.cloud.agent.api.AgentControlAnswer":{"result":
>> true,"wait":0}}]
>> > >> }
>> > >> 2017-01-18 12:17:35,534 DEBUG [c.c.s.StatsCollector]
>> > >> (StatsCollector-4:ctx-37da523c) (logid:aecb957a) AutoScaling
>> > Monitor
>> > >> is running...
>> > >> 2017-01-18 12:17:35,984 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > >> (AsyncJobMgr-Heartbeat-1:ctx-d031d5da) (logid:722c8ae3) Begin
>> > cleanup
>> > >> expired async-jobs
>> > >> 2017-01-18 12:17:35,990 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > >> (AsyncJobMgr-Heartbeat-1:ctx-d031d5da) (logid:722c8ae3) End
>> > cleanup
>> > >> expired async-jobs
>> > >> 2017-01-18 12:17:44,802 DEBUG [c.c.a.m.AgentManagerImpl]
>> > >> (AgentManager-Handler-1:null) (logid:) SeqA 3-438479:
>> Processing
>> > Seq
>> > >> 3-438479: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
>> > >> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand"
>> :{"_proxyVmId":1,
>> > >> "_loadInfo":"{\n
>> > >> \"connections\": []\n}","wait":0}}] }
>> > >> 2017-01-18 12:17:44,863 DEBUG [c.c.a.m.AgentManagerImpl]
>> > >> (AgentManager-Handler-1:null) (logid:) SeqA 3-438479: Sending
>> Seq
>> > >> 3-438479: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1,
>> > Flags:
>> > >> 100010,
>> > >> [{"com.cloud.agent.api.AgentControlAnswer":{"result":
>> true,"wait":0}}]
>> > >> }
>> > >> 2017-01-18 12:17:45,754 DEBUG [c.c.a.m.AgentManagerImpl]
>> > >> (AgentManager-Handler-12:null) (logid:) Ping from 1(iuatc)
>> > >> 2017-01-18 12:17:45,754 DEBUG
>> > [c.c.v.VirtualMachinePowerStateSyncImpl]
>> > >> (AgentManager-Handler-12:null) (logid:) Process host VM state
>> > report
>> > >> from ping process. host: 1
>> > >> 2017-01-18 12:17:45,760 DEBUG
>> > [c.c.v.VirtualMachinePowerStateSyncImpl]
>> > >> (AgentManager-Handler-12:null) (logid:) Process VM state
>> report.
>> > host:
>> > >> 1, number of records in report: 2
>> > >> 2017-01-18 12:17:45,760 DEBUG
>> > [c.c.v.VirtualMachinePowerStateSyncImpl]
>> > >> (AgentManager-Handler-12:null) (logid:) VM state report. host:
>> 1,
>> > vm
>> > >> id:
>> > >> 1, power state: PowerOn
>> > >> 2017-01-18 12:17:45,763 DEBUG
>> > [c.c.v.VirtualMachinePowerStateSyncImpl]
>> > >> (AgentManager-Handler-12:null) (logid:) VM power state does not
>> > >> change, skip DB writing. vm id: 1
>> > >> 2017-01-18 12:17:45,763 DEBUG
>> > [c.c.v.VirtualMachinePowerStateSyncImpl]
>> > >> (AgentManager-Handler-12:null) (logid:) VM state report. host:
>> 1,
>> > vm
>> > >> id:
>> > >> 2, power state: PowerOn
>> > >> 2017-01-18 12:17:45,766 DEBUG
>> > [c.c.v.VirtualMachinePowerStateSyncImpl]
>> > >> (AgentManager-Handler-12:null) (logid:) VM power state does not
>> > >> change, skip DB writing. vm id: 2
>> > >> 2017-01-18 12:17:45,769 DEBUG
>> > [c.c.v.VirtualMachinePowerStateSyncImpl]
>> > >> (AgentManager-Handler-12:null) (logid:) Done with process of VM
>> > state
>> > >> report. host: 1
>> > >> 2017-01-18 12:17:45,985 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > >> (AsyncJobMgr-Heartbeat-1:ctx-b39e7e84) (logid:d0f90660) Begin
>> > cleanup
>> > >> expired async-jobs
>> > >> 2017-01-18 12:17:45,991 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > >> (AsyncJobMgr-Heartbeat-1:ctx-b39e7e84) (logid:d0f90660) End
>> > cleanup
>> > >> expired async-jobs
>> > >> 2017-01-18 12:17:46,046 DEBUG
>> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> > >> (RouterStatusMonitor-1:ctx-36fbb5c2) (logid:d346d004) Found 0
>> > routers
>> > >> to update status.
>> > >> 2017-01-18 12:17:46,047 DEBUG
>> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> > >> (RouterStatusMonitor-1:ctx-36fbb5c2) (logid:d346d004) Found 0
>> VPC
>> > >> networks to update Redundant State.
>> > >> 2017-01-18 12:17:46,049 DEBUG
>> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> > >> (RouterStatusMonitor-1:ctx-36fbb5c2) (logid:d346d004) Found 0
>> > networks
>> > >> to update RvR status.
>> > >> 2017-01-18 12:17:46,097 DEBUG
>> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> > >> (RouterStatusMonitor-1:ctx-16234697) (logid:beb5fec7) Found 0
>> > routers
>> > >> to update status.
>> > >> 2017-01-18 12:17:46,099 DEBUG
>> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> > >> (RouterStatusMonitor-1:ctx-16234697) (logid:beb5fec7) Found 0
>> VPC
>> > >> networks to update Redundant State.
>> > >> 2017-01-18 12:17:46,100 DEBUG
>> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> > >> (RouterStatusMonitor-1:ctx-16234697) (logid:beb5fec7) Found 0
>> > networks
>> > >> to update RvR status.
>> > >> 2017-01-18 12:17:46,201 DEBUG [c.c.a.m.AgentManagerImpl]
>> > >> (AgentManager-Handler-2:null) (logid:) Ping from 2(s-2-VM)
>> > >> 2017-01-18 12:17:47,521 DEBUG [c.c.a.m.AgentManagerImpl]
>> > >> (AgentManager-Handler-14:null) (logid:) Ping from 3(v-1-VM)
>> > >> 2017-01-18 12:17:49,842 DEBUG [c.c.a.m.AgentManagerImpl]
>> > >> (AgentManager-Handler-13:null) (logid:) SeqA 3-438481:
>> Processing
>> > Seq
>> > >> 3-438481: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
>> > >> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand"
>> :{"_proxyVmId":1,
>> > >> "_loadInfo":"{\n
>> > >> \"connections\": []\n}","wait":0}}] }
>> > >> 2017-01-18 12:17:49,897 DEBUG [c.c.a.m.AgentManagerImpl]
>> > >> (AgentManager-Handler-13:null) (logid:) SeqA 3-438481: Sending
>> Seq
>> > >> 3-438481: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1,
>> > Flags:
>> > >> 100010,
>> > >> [{"com.cloud.agent.api.AgentControlAnswer":{"result":
>> true,"wait":0}}]
>> > >> }
>> > >> 2017-01-18 12:17:55,985 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > >> (AsyncJobMgr-Heartbeat-1:ctx-c0db35b5) (logid:c304f172) Begin
>> > cleanup
>> > >> expired async-jobs
>> > >> 2017-01-18 12:17:55,991 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > >> (AsyncJobMgr-Heartbeat-1:ctx-c0db35b5) (logid:c304f172) End
>> > cleanup
>> > >> expired async-jobs
>> > >> 2017-01-18 12:17:59,250 DEBUG [c.c.s.StatsCollector]
>> > >> (StatsCollector-6:ctx-eec84206) (logid:b88e0ce8)
>> VmStatsCollector
>> > is
>> > >> running...
>> > >> 2017-01-18 12:17:59,803 DEBUG [c.c.a.m.AgentManagerImpl]
>> > >> (AgentManager-Handler-11:null) (logid:) SeqA 3-438482:
>> Processing
>> > Seq
>> > >> 3-438482: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
>> > >> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand"
>> :{"_proxyVmId":1,
>> > >> "_loadInfo":"{\n
>> > >> \"connections\": []\n}","wait":0}}] }
>> > >> 2017-01-18 12:17:59,857 DEBUG [c.c.a.m.AgentManagerImpl]
>> > >> (AgentManager-Handler-11:null) (logid:) SeqA 3-438482: Sending
>> Seq
>> > >> 3-438482: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1,
>> > Flags:
>> > >> 100010,
>> > >> [{"com.cloud.agent.api.AgentControlAnswer":{"result":
>> true,"wait":0}}]
>> > >> }
>> > >> 2017-01-18 12:18:00,843 DEBUG
>> > [o.a.c.s.SecondaryStorageManagerImpl]
>> > >> (secstorage-1:ctx-f45be089) (logid:934bd9f7) Zone 1 is ready to
>> > launch
>> > >> secondary storage VM
>> > >> 2017-01-18 12:18:01,094 DEBUG [c.c.c.ConsoleProxyManagerImpl]
>> > >> (consoleproxy-1:ctx-4b11dfe0) (logid:cce52dbb) Zone 1 is ready
>> to
>> > >> launch console proxy
>> > >> 2017-01-18 12:18:03,345 DEBUG [c.c.s.StatsCollector]
>> > >> (StatsCollector-2:ctx-c743d4ee) (logid:c1f62e31)
>> HostStatsCollector
>> > is
>> > >> running...
>> > >> 2017-01-18 12:18:03,406 DEBUG [c.c.a.t.Request]
>> > >> (StatsCollector-2:ctx-c743d4ee) (logid:c1f62e31) Seq
>> > >> 1-6227070909769882805: Received: { Ans: , MgmtId:
>> > 279278805450954,
>> > >> via:
>> > >> 1(iuatc), Ver: v1, Flags: 10, { GetHostStatsAnswer } }
>> > >> 2017-01-18 12:18:04,804 DEBUG [c.c.a.m.AgentManagerImpl]
>> > >> (AgentManager-Handler-9:null) (logid:) SeqA 3-438483:
>> Processing
>> > Seq
>> > >> 3-438483: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
>> > >> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand"
>> :{"_proxyVmId":1,
>> > >> "_loadInfo":"{\n
>> > >> \"connections\": []\n}","wait":0}}] }
>> > >> 2017-01-18 12:18:04,858 DEBUG [c.c.a.m.AgentManagerImpl]
>> > >> (AgentManager-Handler-9:null) (logid:) SeqA 3-438483: Sending
>> Seq
>> > >> 3-438483: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1,
>> > Flags:
>> > >> 100010,
>> > >> [{"com.cloud.agent.api.AgentControlAnswer":{"result":
>> true,"wait":0}}]
>> > >> }
>> > >> 2017-01-18 12:18:05,985 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > >> (AsyncJobMgr-Heartbeat-1:ctx-149380f2) (logid:53d166f9) Begin
>> > cleanup
>> > >> expired async-jobs
>> > >> 2017-01-18 12:18:05,990 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > >> (AsyncJobMgr-Heartbeat-1:ctx-149380f2) (logid:53d166f9) End
>> > cleanup
>> > >> expired async-jobs
>> > >> 2017-01-18 12:18:14,805 DEBUG [c.c.a.m.AgentManagerImpl]
>> > >> (AgentManager-Handler-5:null) (logid:) SeqA 3-438484:
>> Processing
>> > Seq
>> > >> 3-438484: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
>> > >> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand"
>> :{"_proxyVmId":1,
>> > >> "_loadInfo":"{\n
>> > >> \"connections\": []\n}","wait":0}}] }
>> > >> 2017-01-18 12:18:14,951 DEBUG [c.c.a.m.AgentManagerImpl]
>> > >> (AgentManager-Handler-5:null) (logid:) SeqA 3-438484: Sending
>> Seq
>> > >> 3-438484: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1,
>> > Flags:
>> > >> 100010,
>> > >> [{"com.cloud.agent.api.AgentControlAnswer":{"result":
>> true,"wait":0}}]
>> > >> }
>> > >> 2017-01-18 12:18:15,985 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > >> (AsyncJobMgr-Heartbeat-1:ctx-7ccabf83) (logid:2b0f34c4) Begin
>> > cleanup
>> > >> expired async-jobs
>> > >> 2017-01-18 12:18:15,993 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > >> (AsyncJobMgr-Heartbeat-1:ctx-7ccabf83) (logid:2b0f34c4) End
>> > cleanup
>> > >> expired async-jobs
>> > >> 2017-01-18 12:18:16,046 DEBUG
>> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> > >> (RouterStatusMonitor-1:ctx-fc5408a0) (logid:bbd625c1) Found 0
>> > routers
>> > >> to update status.
>> > >> 2017-01-18 12:18:16,048 DEBUG
>> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> > >> (RouterStatusMonitor-1:ctx-fc5408a0) (logid:bbd625c1) Found 0
>> VPC
>> > >> networks to update Redundant State.
>> > >> 2017-01-18 12:18:16,049 DEBUG
>> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> > >> (RouterStatusMonitor-1:ctx-fc5408a0) (logid:bbd625c1) Found 0
>> > networks
>> > >> to update RvR status.
>> > >> 2017-01-18 12:18:16,097 DEBUG
>> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> > >> (RouterStatusMonitor-1:ctx-2c697f6b) (logid:cfa955f2) Found 0
>> > routers
>> > >> to update status.
>> > >> 2017-01-18 12:18:16,099 DEBUG
>> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> > >> (RouterStatusMonitor-1:ctx-2c697f6b) (logid:cfa955f2) Found 0
>> VPC
>> > >> networks to update Redundant State.
>> > >> 2017-01-18 12:18:16,101 DEBUG
>> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> > >> (RouterStatusMonitor-1:ctx-2c697f6b) (logid:cfa955f2) Found 0
>> > networks
>> > >> to update RvR status.
>> > >> 2017-01-18 12:18:24,389 DEBUG [c.c.h.d.HostDaoImpl]
>> > >> (ClusteredAgentManager
>> > >> Timer:ctx-d699e7ed) (logid:e10673e8) Resetting hosts suitable
>> for
>> > >> reconnect
>> > >> 2017-01-18 12:18:24,392 DEBUG [c.c.h.d.HostDaoImpl]
>> > >> (ClusteredAgentManager
>> > >> Timer:ctx-d699e7ed) (logid:e10673e8) Completed resetting hosts
>> > >> suitable for reconnect
>> > >> 2017-01-18 12:18:24,392 DEBUG [c.c.h.d.HostDaoImpl]
>> > >> (ClusteredAgentManager
>> > >> Timer:ctx-d699e7ed) (logid:e10673e8) Acquiring hosts for
>> clusters
>> > >> already owned by this management server
>> > >> 2017-01-18 12:18:24,392 DEBUG [c.c.h.d.HostDaoImpl]
>> > >> (ClusteredAgentManager
>> > >> Timer:ctx-d699e7ed) (logid:e10673e8) Completed acquiring hosts
>> for
>> > >> clusters already owned by this management server
>> > >> 2017-01-18 12:18:24,392 DEBUG [c.c.h.d.HostDaoImpl]
>> > >> (ClusteredAgentManager
>> > >> Timer:ctx-d699e7ed) (logid:e10673e8) Acquiring hosts for
>> clusters
>> > not
>> > >> owned by any management server
>> > >> 2017-01-18 12:18:24,393 DEBUG [c.c.h.d.HostDaoImpl]
>> > >> (ClusteredAgentManager
>> > >> Timer:ctx-d699e7ed) (logid:e10673e8) Completed acquiring hosts
>> for
>> > >> clusters not owned by any management server
>> > >> 2017-01-18 12:18:24,807 DEBUG [c.c.a.m.AgentManagerImpl]
>> > >> (AgentManager-Handler-6:null) (logid:) SeqA 3-438485:
>> Processing
>> > Seq
>> > >> 3-438485: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
>> > >> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand"
>> :{"_proxyVmId":1,
>> > >> "_loadInfo":"{\n
>> > >> \"connections\": []\n}","wait":0}}] }
>> > >> 2017-01-18 12:18:24,870 DEBUG [c.c.a.m.AgentManagerImpl]
>> > >> (AgentManager-Handler-6:null) (logid:) SeqA 3-438485: Sending
>> Seq
>> > >> 3-438485: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1,
>> > Flags:
>> > >> 100010,
>> [{"com.cloud.agent.api.AgentControlAnswer":{"result":tru
>> > >>
>> > >>> Can you please share your management_server logs
>> > >>>
>> > >>> On Tue, Jan 17, 2017 at 11:47 PM,
>> > ì¡°ëÅ\u2019â\u201a¬ÃË\u0153â\u20ac¢
>> > >>> <ca...@renet.kr> wrote:
>> > >>>
>> > >>>> Hi, all
>> > >>>>
>> > >>>> I am facing secondary storage problem.
>> > >>>>
>> > >>>> I am using NFS Server to use primary and secondary storage.
>> > >>>>
>> > >>>> After rebooting the host,
>> > >>>>
>> > >>>> In Dashboard Cloudstack , it says Secondary Storage 0.00 KB /
>> > 0.00
>> > >>>> KB
>> > >>>>
>> > >>>> Primary Storage is okay.
>> > >>>>
>> > >>>>
>> > >>>>
>> > >>>> After all, system VMs are not up. It was working fine before
>> > >>>> rebooting.
>> > >>>>
>> > >>>>
>> > >>>>
>> > >>>> I have tried several times of restart of Management service.
>> > >>>>
>> > >>>>
>> > >>>>
>> > >>>> Any ideas?
>> > >>>>
>> > >>>>
>> > >>>>
>> > >>>> Thanks,
>> > >>>>
>> > >>>>
>> > >>>
>> > >>
>> > >>
>> > >>
>> > >
>> > >
>> > >
>> >
>> >
>> >
>> >
>> >
>> > Dag.Sonstebo@shapeblue.com
>> > www.shapeblue.com
>> > 53 Chandos Place, Covent Garden, London WC2N 4HSUK
>> > @shapeblue
>> >
>> >
>> >
>> >
>>
>>
>>
>
Re: Cloudstack 4.7 Secondary Storage problem
Posted by Vivek Kumar <vi...@indiqus.com>.
try "mount -t nfs 10.8.19.111:/export/secondary/ /tmp/secondary"
*Vivek Kumar*
Virtualization and Cloud Consultant
[image: http://www.indiqus.com/images/logo.jpg] <http://www.indiqus.com/>
*I*ndi*Q*us Technologies Pvt Ltd
A-98, LGF, C.R.Park, New Delhi - 110019
*O* +91 11 4055 1411 | *M* +91 7503460090
www.indiqus.com <http://www.indiqus.com/>
On Wed, Jan 18, 2017 at 5:23 PM, <me...@students.iitmandi.ac.in> wrote:
> Hi,
>
> I tried like you said: here is the result
>
> root@s-2-VM:~# mount -t NFS 10.8.19.111:/export/secondary/ /tmp/secondary/
> mount: unknown filesystem type 'NFS'
> root@s-2-VM:~# mount -t 10.8.19.111:/export/secondary/ /tmp/secondary/
> nfs
> mount: mount point nfs does not exist
>
> I might be running it wrong. Also, i checked the ssvm for the nfs package
> and neither was it there, nor could i download.
> Please help
>
>
> > Hi Merlin,
> >
> > What Paul means is you need to log on to the SSVM, then do something
> like
> > this:
> >
> > mkdir /tmp/secondary
> > mount –t NFS <NFS server IP>:/<NFS/path/here>/ /tmp/secondary
> >
> > This will tell you if your NFS permissions are configured correctly.
> >
> > Regards,
> > Dag Sonstebo
> > Cloud Architect
> > ShapeBlue
> >
> > On 18/01/2017, 09:29, "merlin@students.iitmandi.ac.in"
> > <me...@students.iitmandi.ac.in> wrote:
> >
> > Also when you say mount from ssvm. Does it mean this?
> > Log into ssvm and run "mount":- the reply is down below. I am not
> able
> > to
> > understand much even from the documentation,
> > I am sorry for the lack of full knowledge in this concern.
> >
> >
> > sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
> > proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
> > udev on /dev type devtmpfs
> > (rw,relatime,size=10240k,nr_inodes=62128,mode=755)
> > devpts on /dev/pts type devpts
> > (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
> > tmpfs on /run type tmpfs
> > (rw,nosuid,noexec,relatime,size=50896k,mode=755)
> > /dev/disk/by-uuid/30c81d3d-ee9f-4a88-81c1-5f349b22ba1d on / type
> ext4
> > (rw,relatime,errors=remount-ro,user_xattr,barrier=1,data=ordered)
> > tmpfs on /run/lock type tmpfs
> > (rw,nosuid,nodev,noexec,relatime,size=5120k)
> > tmpfs on /run/shm type tmpfs
> > (rw,nosuid,nodev,noexec,relatime,size=211960k)
> > /dev/vda1 on /boot type ext4
> > (rw,relatime,user_xattr,barrier=1,data=ordered)
> > /dev/vda6 on /home type ext4
> > (rw,relatime,user_xattr,barrier=1,data=ordered)
> > /dev/vda8 on /opt type ext4
> > (rw,relatime,user_xattr,barrier=1,data=ordered)
> > /dev/vda11 on /tmp type ext4
> > (rw,relatime,user_xattr,barrier=1,data=ordered)
> > /dev/vda7 on /usr type ext4
> > (rw,relatime,user_xattr,barrier=1,data=ordered)
> > /dev/vda9 on /var type ext4
> > (rw,relatime,user_xattr,barrier=1,data=ordered)
> > /dev/vda10 on /var/log type ext4
> > (rw,relatime,user_xattr,barrier=1,data=ordered)
> > rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
> >
> > > Hi Merlin,
> > >
> > > To troubleshoot, try mounting the NFS yourself from the SSVM and
> get
> > that
> > > working.
> > >
> > > the likely problem is permissions on the NFS mounts. Either the IP
> > of the
> > > SSVM is not allowed or the permissions don't include no_root_squash
> > (or
> > > equivalent) the SSVM is probably connecting as 'cloud', so
> > connections
> > > have to be mapped to 'root'. Finally the SSVM only reliably
> > supports
> > > NFSv3
> > >
> > >
> > >
> > >
> > > Kind regards,
> > >
> > > Paul Angus
> > >
> > > paul.angus@shapeblue.comÂ
> > > www.shapeblue.com
> > > 53 Chandos Place, Covent Garden, London WC2N 4HSUK
> > > @shapeblue
> > >
> > >
> > >
> > >
> > > -----Original Message-----
> > > From: merlin@students.iitmandi.ac.in
> > > [mailto:merlin@students.iitmandi.ac.in]
> > > Sent: 18 January 2017 08:48
> > > To: users@cloudstack.apache.org
> > > Subject: Re: Cloudstack 4.7 Secondary Storage problem
> > >
> > > And this is the result of check.sh
> > >
> > > 2 packets transmitted, 2 packets received, 0% packet loss
> > round-trip
> > > min/avg/max/stddev = 0.400/0.550/0.701/0.151 ms
> > > Good: Can ping DNS server
> > > ================================================
> > > Good: DNS resolves download.cloud.com
> > > ================================================
> > > ERROR: NFS is not currently mounted
> > > Try manually mounting from inside the VM NFS server is 10.8.0.1
> > PING
> > > 10.8.0.1 (10.8.0.1): 48 data bytes
> > > 56 bytes from 10.8.0.1: icmp_seq=0 ttl=64 time=0.456 ms
> > > 56 bytes from 10.8.0.1: icmp_seq=1 ttl=64 time=0.708 ms
> > > --- 10.8.0.1 ping statistics ---
> > > 2 packets transmitted, 2 packets received, 0% packet loss
> > round-trip
> > > min/avg/max/stddev = 0.456/0.582/0.708/0.126 ms
> > > Good: Can ping nfs server
> > > ================================================
> > > Management server is 10.8.19.111. Checking connectivity.
> > > Good: Can connect to management server port 8250
> > > ================================================
> > > Good: Java process is running
> > > ================================================
> > > Tests Complete. Look for ERROR or WARNING above.
> > >
> > >
> > > The problem is that while installation I did mount nfs?
> > > I dont understand what else to do next
> > >
> > >
> > >
> > >> Hi, even I am facing the same problem.. and here are the logs from
> > >> today:
> > >>
> > >>
> > >>
> > >> 2017-01-18 12:16:46,100 DEBUG
> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> > >> (RouterStatusMonitor-1:ctx-bc5cb4cd) (logid:0ffac658) Found 0
> > networks
> > >> to update RvR status.
> > >> 2017-01-18 12:16:46,200 DEBUG [c.c.a.m.AgentManagerImpl]
> > >> (AgentManager-Handler-11:null) (logid:) Ping from 2(s-2-VM)
> > >> 2017-01-18 12:16:47,521 DEBUG [c.c.a.m.AgentManagerImpl]
> > >> (AgentManager-Handler-15:null) (logid:) Ping from 3(v-1-VM)
> > >> 2017-01-18 12:16:49,834 DEBUG [c.c.a.m.AgentManagerImpl]
> > >> (AgentManager-Handler-9:null) (logid:) SeqA 3-438472: Processing
> > Seq
> > >> 3-438472: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
> > >> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand"
> :{"_proxyVmId":1,
> > >> "_loadInfo":"{\n
> > >> \"connections\": []\n}","wait":0}}] }
> > >> 2017-01-18 12:16:49,887 DEBUG [c.c.a.m.AgentManagerImpl]
> > >> (AgentManager-Handler-9:null) (logid:) SeqA 3-438472: Sending Seq
> > >> 3-438472: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1,
> > Flags:
> > >> 100010,
> > >> [{"com.cloud.agent.api.AgentControlAnswer":{"result":
> true,"wait":0}}]
> > >> }
> > >> 2017-01-18 12:16:54,390 DEBUG [c.c.h.d.HostDaoImpl]
> > >> (ClusteredAgentManager
> > >> Timer:ctx-8464e4b3) (logid:1f15c721) Resetting hosts suitable for
> > >> reconnect
> > >> 2017-01-18 12:16:54,392 DEBUG [c.c.h.d.HostDaoImpl]
> > >> (ClusteredAgentManager
> > >> Timer:ctx-8464e4b3) (logid:1f15c721) Completed resetting hosts
> > >> suitable for reconnect
> > >> 2017-01-18 12:16:54,392 DEBUG [c.c.h.d.HostDaoImpl]
> > >> (ClusteredAgentManager
> > >> Timer:ctx-8464e4b3) (logid:1f15c721) Acquiring hosts for clusters
> > >> already owned by this management server
> > >> 2017-01-18 12:16:54,393 DEBUG [c.c.h.d.HostDaoImpl]
> > >> (ClusteredAgentManager
> > >> Timer:ctx-8464e4b3) (logid:1f15c721) Completed acquiring hosts for
> > >> clusters already owned by this management server
> > >> 2017-01-18 12:16:54,393 DEBUG [c.c.h.d.HostDaoImpl]
> > >> (ClusteredAgentManager
> > >> Timer:ctx-8464e4b3) (logid:1f15c721) Acquiring hosts for clusters
> > not
> > >> owned by any management server
> > >> 2017-01-18 12:16:54,394 DEBUG [c.c.h.d.HostDaoImpl]
> > >> (ClusteredAgentManager
> > >> Timer:ctx-8464e4b3) (logid:1f15c721) Completed acquiring hosts for
> > >> clusters not owned by any management server
> > >> 2017-01-18 12:16:54,795 DEBUG [c.c.a.m.AgentManagerImpl]
> > >> (AgentManager-Handler-5:null) (logid:) SeqA 3-438473: Processing
> > Seq
> > >> 3-438473: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
> > >> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand"
> :{"_proxyVmId":1,
> > >> "_loadInfo":"{\n
> > >> \"connections\": []\n}","wait":0}}] }
> > >> 2017-01-18 12:16:54,854 DEBUG [c.c.a.m.AgentManagerImpl]
> > >> (AgentManager-Handler-5:null) (logid:) SeqA 3-438473: Sending Seq
> > >> 3-438473: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1,
> > Flags:
> > >> 100010,
> > >> [{"com.cloud.agent.api.AgentControlAnswer":{"result":
> true,"wait":0}}]
> > >> }
> > >> 2017-01-18 12:16:55,984 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> > >> (AsyncJobMgr-Heartbeat-1:ctx-d0469453) (logid:af5d4f47) Begin
> > cleanup
> > >> expired async-jobs
> > >> 2017-01-18 12:16:55,991 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> > >> (AsyncJobMgr-Heartbeat-1:ctx-d0469453) (logid:af5d4f47) End
> > cleanup
> > >> expired async-jobs
> > >> 2017-01-18 12:16:59,245 DEBUG [c.c.s.StatsCollector]
> > >> (StatsCollector-1:ctx-68069dff) (logid:a7a6974d) VmStatsCollector
> > is
> > >> running...
> > >> 2017-01-18 12:17:00,841 DEBUG
> > [o.a.c.s.SecondaryStorageManagerImpl]
> > >> (secstorage-1:ctx-4e101aa5) (logid:65f1f0cb) Zone 1 is ready to
> > launch
> > >> secondary storage VM
> > >> 2017-01-18 12:17:01,095 DEBUG [c.c.c.ConsoleProxyManagerImpl]
> > >> (consoleproxy-1:ctx-393828fa) (logid:7b3d0491) Zone 1 is ready to
> > >> launch console proxy
> > >> 2017-01-18 12:17:03,283 DEBUG [c.c.s.StatsCollector]
> > >> (StatsCollector-3:ctx-cc1f58a1) (logid:3ae41340)
> HostStatsCollector
> > is
> > >> running...
> > >> 2017-01-18 12:17:03,343 DEBUG [c.c.a.t.Request]
> > >> (StatsCollector-3:ctx-cc1f58a1) (logid:3ae41340) Seq
> > >> 1-6227070909769882804: Received: { Ans: , MgmtId:
> > 279278805450954,
> > >> via:
> > >> 1(iuatc), Ver: v1, Flags: 10, { GetHostStatsAnswer } }
> > >> 2017-01-18 12:17:04,796 DEBUG [c.c.a.m.AgentManagerImpl]
> > >> (AgentManager-Handler-10:null) (logid:) SeqA 3-438474: Processing
> > Seq
> > >> 3-438474: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
> > >> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand"
> :{"_proxyVmId":1,
> > >> "_loadInfo":"{\n
> > >> \"connections\": []\n}","wait":0}}] }
> > >> 2017-01-18 12:17:04,847 DEBUG [c.c.a.m.AgentManagerImpl]
> > >> (AgentManager-Handler-10:null) (logid:) SeqA 3-438474: Sending Seq
> > >> 3-438474: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1,
> > Flags:
> > >> 100010,
> > >> [{"com.cloud.agent.api.AgentControlAnswer":{"result":
> true,"wait":0}}]
> > >> }
> > >> 2017-01-18 12:17:05,985 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> > >> (AsyncJobMgr-Heartbeat-1:ctx-7f0de3b8) (logid:e9bbd63a) Begin
> > cleanup
> > >> expired async-jobs
> > >> 2017-01-18 12:17:05,991 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> > >> (AsyncJobMgr-Heartbeat-1:ctx-7f0de3b8) (logid:e9bbd63a) End
> > cleanup
> > >> expired async-jobs
> > >> 2017-01-18 12:17:14,798 DEBUG [c.c.a.m.AgentManagerImpl]
> > >> (AgentManager-Handler-8:null) (logid:) SeqA 3-438475: Processing
> > Seq
> > >> 3-438475: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
> > >> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand"
> :{"_proxyVmId":1,
> > >> "_loadInfo":"{\n
> > >> \"connections\": []\n}","wait":0}}] }
> > >> 2017-01-18 12:17:14,857 DEBUG [c.c.a.m.AgentManagerImpl]
> > >> (AgentManager-Handler-8:null) (logid:) SeqA 3-438475: Sending Seq
> > >> 3-438475: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1,
> > Flags:
> > >> 100010,
> > >> [{"com.cloud.agent.api.AgentControlAnswer":{"result":
> true,"wait":0}}]
> > >> }
> > >> 2017-01-18 12:17:15,984 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> > >> (AsyncJobMgr-Heartbeat-1:ctx-5682fe3c) (logid:901efb52) Begin
> > cleanup
> > >> expired async-jobs
> > >> 2017-01-18 12:17:15,990 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> > >> (AsyncJobMgr-Heartbeat-1:ctx-5682fe3c) (logid:901efb52) End
> > cleanup
> > >> expired async-jobs
> > >> 2017-01-18 12:17:16,046 DEBUG
> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> > >> (RouterStatusMonitor-1:ctx-7ad0b18a) (logid:b5b19f2b) Found 0
> > routers
> > >> to update status.
> > >> 2017-01-18 12:17:16,047 DEBUG
> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> > >> (RouterStatusMonitor-1:ctx-7ad0b18a) (logid:b5b19f2b) Found 0 VPC
> > >> networks to update Redundant State.
> > >> 2017-01-18 12:17:16,049 DEBUG
> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> > >> (RouterStatusMonitor-1:ctx-7ad0b18a) (logid:b5b19f2b) Found 0
> > networks
> > >> to update RvR status.
> > >> 2017-01-18 12:17:16,097 DEBUG
> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> > >> (RouterStatusMonitor-1:ctx-10a8aefd) (logid:b09ffa05) Found 0
> > routers
> > >> to update status.
> > >> 2017-01-18 12:17:16,099 DEBUG
> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> > >> (RouterStatusMonitor-1:ctx-10a8aefd) (logid:b09ffa05) Found 0 VPC
> > >> networks to update Redundant State.
> > >> 2017-01-18 12:17:16,101 DEBUG
> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> > >> (RouterStatusMonitor-1:ctx-10a8aefd) (logid:b09ffa05) Found 0
> > networks
> > >> to update RvR status.
> > >> 2017-01-18 12:17:19,799 DEBUG [c.c.a.m.AgentManagerImpl]
> > >> (AgentManager-Handler-3:null) (logid:) SeqA 3-438476: Processing
> > Seq
> > >> 3-438476: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
> > >> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand"
> :{"_proxyVmId":1,
> > >> "_loadInfo":"{\n
> > >> \"connections\": []\n}","wait":0}}] }
> > >> 2017-01-18 12:17:19,925 DEBUG [c.c.a.m.AgentManagerImpl]
> > >> (AgentManager-Handler-3:null) (logid:) SeqA 3-438476: Sending Seq
> > >> 3-438476: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1,
> > Flags:
> > >> 100010,
> > >> [{"com.cloud.agent.api.AgentControlAnswer":{"result":
> true,"wait":0}}]
> > >> }
> > >> 2017-01-18 12:17:25,985 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> > >> (AsyncJobMgr-Heartbeat-1:ctx-a4c8ceb4) (logid:d0d85ed6) Begin
> > cleanup
> > >> expired async-jobs
> > >> 2017-01-18 12:17:25,991 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> > >> (AsyncJobMgr-Heartbeat-1:ctx-a4c8ceb4) (logid:d0d85ed6) End
> > cleanup
> > >> expired async-jobs
> > >> 2017-01-18 12:17:29,800 DEBUG [c.c.a.m.AgentManagerImpl]
> > >> (AgentManager-Handler-7:null) (logid:) SeqA 3-438477: Processing
> > Seq
> > >> 3-438477: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
> > >> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand"
> :{"_proxyVmId":1,
> > >> "_loadInfo":"{\n
> > >> \"connections\": []\n}","wait":0}}] }
> > >> 2017-01-18 12:17:29,860 DEBUG [c.c.a.m.AgentManagerImpl]
> > >> (AgentManager-Handler-7:null) (logid:) SeqA 3-438477: Sending Seq
> > >> 3-438477: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1,
> > Flags:
> > >> 100010,
> > >> [{"com.cloud.agent.api.AgentControlAnswer":{"result":
> true,"wait":0}}]
> > >> }
> > >> 2017-01-18 12:17:30,844 DEBUG
> > [o.a.c.s.SecondaryStorageManagerImpl]
> > >> (secstorage-1:ctx-f2d92c7a) (logid:6bcbea1f) Zone 1 is ready to
> > launch
> > >> secondary storage VM
> > >> 2017-01-18 12:17:31,095 DEBUG [c.c.c.ConsoleProxyManagerImpl]
> > >> (consoleproxy-1:ctx-5c5c1698) (logid:a199e87c) Zone 1 is ready to
> > >> launch console proxy
> > >> 2017-01-18 12:17:34,801 DEBUG [c.c.a.m.AgentManagerImpl]
> > >> (AgentManager-Handler-4:null) (logid:) SeqA 3-438478: Processing
> > Seq
> > >> 3-438478: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
> > >> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand"
> :{"_proxyVmId":1,
> > >> "_loadInfo":"{\n
> > >> \"connections\": []\n}","wait":0}}] }
> > >> 2017-01-18 12:17:34,853 DEBUG [c.c.a.m.AgentManagerImpl]
> > >> (AgentManager-Handler-4:null) (logid:) SeqA 3-438478: Sending Seq
> > >> 3-438478: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1,
> > Flags:
> > >> 100010,
> > >> [{"com.cloud.agent.api.AgentControlAnswer":{"result":
> true,"wait":0}}]
> > >> }
> > >> 2017-01-18 12:17:35,534 DEBUG [c.c.s.StatsCollector]
> > >> (StatsCollector-4:ctx-37da523c) (logid:aecb957a) AutoScaling
> > Monitor
> > >> is running...
> > >> 2017-01-18 12:17:35,984 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> > >> (AsyncJobMgr-Heartbeat-1:ctx-d031d5da) (logid:722c8ae3) Begin
> > cleanup
> > >> expired async-jobs
> > >> 2017-01-18 12:17:35,990 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> > >> (AsyncJobMgr-Heartbeat-1:ctx-d031d5da) (logid:722c8ae3) End
> > cleanup
> > >> expired async-jobs
> > >> 2017-01-18 12:17:44,802 DEBUG [c.c.a.m.AgentManagerImpl]
> > >> (AgentManager-Handler-1:null) (logid:) SeqA 3-438479: Processing
> > Seq
> > >> 3-438479: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
> > >> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand"
> :{"_proxyVmId":1,
> > >> "_loadInfo":"{\n
> > >> \"connections\": []\n}","wait":0}}] }
> > >> 2017-01-18 12:17:44,863 DEBUG [c.c.a.m.AgentManagerImpl]
> > >> (AgentManager-Handler-1:null) (logid:) SeqA 3-438479: Sending Seq
> > >> 3-438479: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1,
> > Flags:
> > >> 100010,
> > >> [{"com.cloud.agent.api.AgentControlAnswer":{"result":
> true,"wait":0}}]
> > >> }
> > >> 2017-01-18 12:17:45,754 DEBUG [c.c.a.m.AgentManagerImpl]
> > >> (AgentManager-Handler-12:null) (logid:) Ping from 1(iuatc)
> > >> 2017-01-18 12:17:45,754 DEBUG
> > [c.c.v.VirtualMachinePowerStateSyncImpl]
> > >> (AgentManager-Handler-12:null) (logid:) Process host VM state
> > report
> > >> from ping process. host: 1
> > >> 2017-01-18 12:17:45,760 DEBUG
> > [c.c.v.VirtualMachinePowerStateSyncImpl]
> > >> (AgentManager-Handler-12:null) (logid:) Process VM state report.
> > host:
> > >> 1, number of records in report: 2
> > >> 2017-01-18 12:17:45,760 DEBUG
> > [c.c.v.VirtualMachinePowerStateSyncImpl]
> > >> (AgentManager-Handler-12:null) (logid:) VM state report. host: 1,
> > vm
> > >> id:
> > >> 1, power state: PowerOn
> > >> 2017-01-18 12:17:45,763 DEBUG
> > [c.c.v.VirtualMachinePowerStateSyncImpl]
> > >> (AgentManager-Handler-12:null) (logid:) VM power state does not
> > >> change, skip DB writing. vm id: 1
> > >> 2017-01-18 12:17:45,763 DEBUG
> > [c.c.v.VirtualMachinePowerStateSyncImpl]
> > >> (AgentManager-Handler-12:null) (logid:) VM state report. host: 1,
> > vm
> > >> id:
> > >> 2, power state: PowerOn
> > >> 2017-01-18 12:17:45,766 DEBUG
> > [c.c.v.VirtualMachinePowerStateSyncImpl]
> > >> (AgentManager-Handler-12:null) (logid:) VM power state does not
> > >> change, skip DB writing. vm id: 2
> > >> 2017-01-18 12:17:45,769 DEBUG
> > [c.c.v.VirtualMachinePowerStateSyncImpl]
> > >> (AgentManager-Handler-12:null) (logid:) Done with process of VM
> > state
> > >> report. host: 1
> > >> 2017-01-18 12:17:45,985 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> > >> (AsyncJobMgr-Heartbeat-1:ctx-b39e7e84) (logid:d0f90660) Begin
> > cleanup
> > >> expired async-jobs
> > >> 2017-01-18 12:17:45,991 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> > >> (AsyncJobMgr-Heartbeat-1:ctx-b39e7e84) (logid:d0f90660) End
> > cleanup
> > >> expired async-jobs
> > >> 2017-01-18 12:17:46,046 DEBUG
> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> > >> (RouterStatusMonitor-1:ctx-36fbb5c2) (logid:d346d004) Found 0
> > routers
> > >> to update status.
> > >> 2017-01-18 12:17:46,047 DEBUG
> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> > >> (RouterStatusMonitor-1:ctx-36fbb5c2) (logid:d346d004) Found 0 VPC
> > >> networks to update Redundant State.
> > >> 2017-01-18 12:17:46,049 DEBUG
> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> > >> (RouterStatusMonitor-1:ctx-36fbb5c2) (logid:d346d004) Found 0
> > networks
> > >> to update RvR status.
> > >> 2017-01-18 12:17:46,097 DEBUG
> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> > >> (RouterStatusMonitor-1:ctx-16234697) (logid:beb5fec7) Found 0
> > routers
> > >> to update status.
> > >> 2017-01-18 12:17:46,099 DEBUG
> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> > >> (RouterStatusMonitor-1:ctx-16234697) (logid:beb5fec7) Found 0 VPC
> > >> networks to update Redundant State.
> > >> 2017-01-18 12:17:46,100 DEBUG
> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> > >> (RouterStatusMonitor-1:ctx-16234697) (logid:beb5fec7) Found 0
> > networks
> > >> to update RvR status.
> > >> 2017-01-18 12:17:46,201 DEBUG [c.c.a.m.AgentManagerImpl]
> > >> (AgentManager-Handler-2:null) (logid:) Ping from 2(s-2-VM)
> > >> 2017-01-18 12:17:47,521 DEBUG [c.c.a.m.AgentManagerImpl]
> > >> (AgentManager-Handler-14:null) (logid:) Ping from 3(v-1-VM)
> > >> 2017-01-18 12:17:49,842 DEBUG [c.c.a.m.AgentManagerImpl]
> > >> (AgentManager-Handler-13:null) (logid:) SeqA 3-438481: Processing
> > Seq
> > >> 3-438481: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
> > >> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand"
> :{"_proxyVmId":1,
> > >> "_loadInfo":"{\n
> > >> \"connections\": []\n}","wait":0}}] }
> > >> 2017-01-18 12:17:49,897 DEBUG [c.c.a.m.AgentManagerImpl]
> > >> (AgentManager-Handler-13:null) (logid:) SeqA 3-438481: Sending Seq
> > >> 3-438481: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1,
> > Flags:
> > >> 100010,
> > >> [{"com.cloud.agent.api.AgentControlAnswer":{"result":
> true,"wait":0}}]
> > >> }
> > >> 2017-01-18 12:17:55,985 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> > >> (AsyncJobMgr-Heartbeat-1:ctx-c0db35b5) (logid:c304f172) Begin
> > cleanup
> > >> expired async-jobs
> > >> 2017-01-18 12:17:55,991 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> > >> (AsyncJobMgr-Heartbeat-1:ctx-c0db35b5) (logid:c304f172) End
> > cleanup
> > >> expired async-jobs
> > >> 2017-01-18 12:17:59,250 DEBUG [c.c.s.StatsCollector]
> > >> (StatsCollector-6:ctx-eec84206) (logid:b88e0ce8) VmStatsCollector
> > is
> > >> running...
> > >> 2017-01-18 12:17:59,803 DEBUG [c.c.a.m.AgentManagerImpl]
> > >> (AgentManager-Handler-11:null) (logid:) SeqA 3-438482: Processing
> > Seq
> > >> 3-438482: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
> > >> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand"
> :{"_proxyVmId":1,
> > >> "_loadInfo":"{\n
> > >> \"connections\": []\n}","wait":0}}] }
> > >> 2017-01-18 12:17:59,857 DEBUG [c.c.a.m.AgentManagerImpl]
> > >> (AgentManager-Handler-11:null) (logid:) SeqA 3-438482: Sending Seq
> > >> 3-438482: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1,
> > Flags:
> > >> 100010,
> > >> [{"com.cloud.agent.api.AgentControlAnswer":{"result":
> true,"wait":0}}]
> > >> }
> > >> 2017-01-18 12:18:00,843 DEBUG
> > [o.a.c.s.SecondaryStorageManagerImpl]
> > >> (secstorage-1:ctx-f45be089) (logid:934bd9f7) Zone 1 is ready to
> > launch
> > >> secondary storage VM
> > >> 2017-01-18 12:18:01,094 DEBUG [c.c.c.ConsoleProxyManagerImpl]
> > >> (consoleproxy-1:ctx-4b11dfe0) (logid:cce52dbb) Zone 1 is ready to
> > >> launch console proxy
> > >> 2017-01-18 12:18:03,345 DEBUG [c.c.s.StatsCollector]
> > >> (StatsCollector-2:ctx-c743d4ee) (logid:c1f62e31)
> HostStatsCollector
> > is
> > >> running...
> > >> 2017-01-18 12:18:03,406 DEBUG [c.c.a.t.Request]
> > >> (StatsCollector-2:ctx-c743d4ee) (logid:c1f62e31) Seq
> > >> 1-6227070909769882805: Received: { Ans: , MgmtId:
> > 279278805450954,
> > >> via:
> > >> 1(iuatc), Ver: v1, Flags: 10, { GetHostStatsAnswer } }
> > >> 2017-01-18 12:18:04,804 DEBUG [c.c.a.m.AgentManagerImpl]
> > >> (AgentManager-Handler-9:null) (logid:) SeqA 3-438483: Processing
> > Seq
> > >> 3-438483: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
> > >> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand"
> :{"_proxyVmId":1,
> > >> "_loadInfo":"{\n
> > >> \"connections\": []\n}","wait":0}}] }
> > >> 2017-01-18 12:18:04,858 DEBUG [c.c.a.m.AgentManagerImpl]
> > >> (AgentManager-Handler-9:null) (logid:) SeqA 3-438483: Sending Seq
> > >> 3-438483: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1,
> > Flags:
> > >> 100010,
> > >> [{"com.cloud.agent.api.AgentControlAnswer":{"result":
> true,"wait":0}}]
> > >> }
> > >> 2017-01-18 12:18:05,985 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> > >> (AsyncJobMgr-Heartbeat-1:ctx-149380f2) (logid:53d166f9) Begin
> > cleanup
> > >> expired async-jobs
> > >> 2017-01-18 12:18:05,990 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> > >> (AsyncJobMgr-Heartbeat-1:ctx-149380f2) (logid:53d166f9) End
> > cleanup
> > >> expired async-jobs
> > >> 2017-01-18 12:18:14,805 DEBUG [c.c.a.m.AgentManagerImpl]
> > >> (AgentManager-Handler-5:null) (logid:) SeqA 3-438484: Processing
> > Seq
> > >> 3-438484: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
> > >> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand"
> :{"_proxyVmId":1,
> > >> "_loadInfo":"{\n
> > >> \"connections\": []\n}","wait":0}}] }
> > >> 2017-01-18 12:18:14,951 DEBUG [c.c.a.m.AgentManagerImpl]
> > >> (AgentManager-Handler-5:null) (logid:) SeqA 3-438484: Sending Seq
> > >> 3-438484: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1,
> > Flags:
> > >> 100010,
> > >> [{"com.cloud.agent.api.AgentControlAnswer":{"result":
> true,"wait":0}}]
> > >> }
> > >> 2017-01-18 12:18:15,985 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> > >> (AsyncJobMgr-Heartbeat-1:ctx-7ccabf83) (logid:2b0f34c4) Begin
> > cleanup
> > >> expired async-jobs
> > >> 2017-01-18 12:18:15,993 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> > >> (AsyncJobMgr-Heartbeat-1:ctx-7ccabf83) (logid:2b0f34c4) End
> > cleanup
> > >> expired async-jobs
> > >> 2017-01-18 12:18:16,046 DEBUG
> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> > >> (RouterStatusMonitor-1:ctx-fc5408a0) (logid:bbd625c1) Found 0
> > routers
> > >> to update status.
> > >> 2017-01-18 12:18:16,048 DEBUG
> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> > >> (RouterStatusMonitor-1:ctx-fc5408a0) (logid:bbd625c1) Found 0 VPC
> > >> networks to update Redundant State.
> > >> 2017-01-18 12:18:16,049 DEBUG
> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> > >> (RouterStatusMonitor-1:ctx-fc5408a0) (logid:bbd625c1) Found 0
> > networks
> > >> to update RvR status.
> > >> 2017-01-18 12:18:16,097 DEBUG
> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> > >> (RouterStatusMonitor-1:ctx-2c697f6b) (logid:cfa955f2) Found 0
> > routers
> > >> to update status.
> > >> 2017-01-18 12:18:16,099 DEBUG
> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> > >> (RouterStatusMonitor-1:ctx-2c697f6b) (logid:cfa955f2) Found 0 VPC
> > >> networks to update Redundant State.
> > >> 2017-01-18 12:18:16,101 DEBUG
> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> > >> (RouterStatusMonitor-1:ctx-2c697f6b) (logid:cfa955f2) Found 0
> > networks
> > >> to update RvR status.
> > >> 2017-01-18 12:18:24,389 DEBUG [c.c.h.d.HostDaoImpl]
> > >> (ClusteredAgentManager
> > >> Timer:ctx-d699e7ed) (logid:e10673e8) Resetting hosts suitable for
> > >> reconnect
> > >> 2017-01-18 12:18:24,392 DEBUG [c.c.h.d.HostDaoImpl]
> > >> (ClusteredAgentManager
> > >> Timer:ctx-d699e7ed) (logid:e10673e8) Completed resetting hosts
> > >> suitable for reconnect
> > >> 2017-01-18 12:18:24,392 DEBUG [c.c.h.d.HostDaoImpl]
> > >> (ClusteredAgentManager
> > >> Timer:ctx-d699e7ed) (logid:e10673e8) Acquiring hosts for clusters
> > >> already owned by this management server
> > >> 2017-01-18 12:18:24,392 DEBUG [c.c.h.d.HostDaoImpl]
> > >> (ClusteredAgentManager
> > >> Timer:ctx-d699e7ed) (logid:e10673e8) Completed acquiring hosts for
> > >> clusters already owned by this management server
> > >> 2017-01-18 12:18:24,392 DEBUG [c.c.h.d.HostDaoImpl]
> > >> (ClusteredAgentManager
> > >> Timer:ctx-d699e7ed) (logid:e10673e8) Acquiring hosts for clusters
> > not
> > >> owned by any management server
> > >> 2017-01-18 12:18:24,393 DEBUG [c.c.h.d.HostDaoImpl]
> > >> (ClusteredAgentManager
> > >> Timer:ctx-d699e7ed) (logid:e10673e8) Completed acquiring hosts for
> > >> clusters not owned by any management server
> > >> 2017-01-18 12:18:24,807 DEBUG [c.c.a.m.AgentManagerImpl]
> > >> (AgentManager-Handler-6:null) (logid:) SeqA 3-438485: Processing
> > Seq
> > >> 3-438485: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
> > >> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand"
> :{"_proxyVmId":1,
> > >> "_loadInfo":"{\n
> > >> \"connections\": []\n}","wait":0}}] }
> > >> 2017-01-18 12:18:24,870 DEBUG [c.c.a.m.AgentManagerImpl]
> > >> (AgentManager-Handler-6:null) (logid:) SeqA 3-438485: Sending Seq
> > >> 3-438485: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1,
> > Flags:
> > >> 100010, [{"com.cloud.agent.api.AgentControlAnswer":{"result":tru
> > >>
> > >>> Can you please share your management_server logs
> > >>>
> > >>> On Tue, Jan 17, 2017 at 11:47 PM,
> > 조대Ø•
> > >>> <ca...@renet.kr> wrote:
> > >>>
> > >>>> Hi, all
> > >>>>
> > >>>> I am facing secondary storage problem.
> > >>>>
> > >>>> I am using NFS Server to use primary and secondary storage.
> > >>>>
> > >>>> After rebooting the host,
> > >>>>
> > >>>> In Dashboard Cloudstack , it says Secondary Storage 0.00 KB /
> > 0.00
> > >>>> KB
> > >>>>
> > >>>> Primary Storage is okay.
> > >>>>
> > >>>>
> > >>>>
> > >>>> After all, system VMs are not up. It was working fine before
> > >>>> rebooting.
> > >>>>
> > >>>>
> > >>>>
> > >>>> I have tried several times of restart of Management service.
> > >>>>
> > >>>>
> > >>>>
> > >>>> Any ideas?
> > >>>>
> > >>>>
> > >>>>
> > >>>> Thanks,
> > >>>>
> > >>>>
> > >>>
> > >>
> > >>
> > >>
> > >
> > >
> > >
> >
> >
> >
> >
> >
> > Dag.Sonstebo@shapeblue.com
> > www.shapeblue.com
> > 53 Chandos Place, Covent Garden, London WC2N 4HSUK
> > @shapeblue
> >
> >
> >
> >
>
>
>
Re: Cloudstack 4.7 Secondary Storage problem
Posted by me...@students.iitmandi.ac.in.
Hi,
I tried like you said: here is the result
root@s-2-VM:~# mount -t NFS 10.8.19.111:/export/secondary/ /tmp/secondary/
mount: unknown filesystem type 'NFS'
root@s-2-VM:~# mount -t 10.8.19.111:/export/secondary/ /tmp/secondary/ nfs
mount: mount point nfs does not exist
I might be running it wrong. Also, i checked the ssvm for the nfs package
and neither was it there, nor could i download.
Please help
> Hi Merlin,
>
> What Paul means is you need to log on to the SSVM, then do something like
> this:
>
> mkdir /tmp/secondary
> mount \u2013t NFS <NFS server IP>:/<NFS/path/here>/ /tmp/secondary
>
> This will tell you if your NFS permissions are configured correctly.
>
> Regards,
> Dag Sonstebo
> Cloud Architect
> ShapeBlue
>
> On 18/01/2017, 09:29, "merlin@students.iitmandi.ac.in"
> <me...@students.iitmandi.ac.in> wrote:
>
> Also when you say mount from ssvm. Does it mean this?
> Log into ssvm and run "mount":- the reply is down below. I am not able
> to
> understand much even from the documentation,
> I am sorry for the lack of full knowledge in this concern.
>
>
> sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
> proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
> udev on /dev type devtmpfs
> (rw,relatime,size=10240k,nr_inodes=62128,mode=755)
> devpts on /dev/pts type devpts
> (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
> tmpfs on /run type tmpfs
> (rw,nosuid,noexec,relatime,size=50896k,mode=755)
> /dev/disk/by-uuid/30c81d3d-ee9f-4a88-81c1-5f349b22ba1d on / type ext4
> (rw,relatime,errors=remount-ro,user_xattr,barrier=1,data=ordered)
> tmpfs on /run/lock type tmpfs
> (rw,nosuid,nodev,noexec,relatime,size=5120k)
> tmpfs on /run/shm type tmpfs
> (rw,nosuid,nodev,noexec,relatime,size=211960k)
> /dev/vda1 on /boot type ext4
> (rw,relatime,user_xattr,barrier=1,data=ordered)
> /dev/vda6 on /home type ext4
> (rw,relatime,user_xattr,barrier=1,data=ordered)
> /dev/vda8 on /opt type ext4
> (rw,relatime,user_xattr,barrier=1,data=ordered)
> /dev/vda11 on /tmp type ext4
> (rw,relatime,user_xattr,barrier=1,data=ordered)
> /dev/vda7 on /usr type ext4
> (rw,relatime,user_xattr,barrier=1,data=ordered)
> /dev/vda9 on /var type ext4
> (rw,relatime,user_xattr,barrier=1,data=ordered)
> /dev/vda10 on /var/log type ext4
> (rw,relatime,user_xattr,barrier=1,data=ordered)
> rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
>
> > Hi Merlin,
> >
> > To troubleshoot, try mounting the NFS yourself from the SSVM and get
> that
> > working.
> >
> > the likely problem is permissions on the NFS mounts. Either the IP
> of the
> > SSVM is not allowed or the permissions don't include no_root_squash
> (or
> > equivalent) the SSVM is probably connecting as 'cloud', so
> connections
> > have to be mapped to 'root'. Finally the SSVM only reliably
> supports
> > NFSv3
> >
> >
> >
> >
> > Kind regards,
> >
> > Paul Angus
> >
> > paul.angus@shapeblue.comÂ
> > www.shapeblue.com
> > 53 Chandos Place, Covent Garden, London WC2N 4HSUK
> > @shapeblue
> >
> >
> >
> >
> > -----Original Message-----
> > From: merlin@students.iitmandi.ac.in
> > [mailto:merlin@students.iitmandi.ac.in]
> > Sent: 18 January 2017 08:48
> > To: users@cloudstack.apache.org
> > Subject: Re: Cloudstack 4.7 Secondary Storage problem
> >
> > And this is the result of check.sh
> >
> > 2 packets transmitted, 2 packets received, 0% packet loss
> round-trip
> > min/avg/max/stddev = 0.400/0.550/0.701/0.151 ms
> > Good: Can ping DNS server
> > ================================================
> > Good: DNS resolves download.cloud.com
> > ================================================
> > ERROR: NFS is not currently mounted
> > Try manually mounting from inside the VM NFS server is 10.8.0.1
> PING
> > 10.8.0.1 (10.8.0.1): 48 data bytes
> > 56 bytes from 10.8.0.1: icmp_seq=0 ttl=64 time=0.456 ms
> > 56 bytes from 10.8.0.1: icmp_seq=1 ttl=64 time=0.708 ms
> > --- 10.8.0.1 ping statistics ---
> > 2 packets transmitted, 2 packets received, 0% packet loss
> round-trip
> > min/avg/max/stddev = 0.456/0.582/0.708/0.126 ms
> > Good: Can ping nfs server
> > ================================================
> > Management server is 10.8.19.111. Checking connectivity.
> > Good: Can connect to management server port 8250
> > ================================================
> > Good: Java process is running
> > ================================================
> > Tests Complete. Look for ERROR or WARNING above.
> >
> >
> > The problem is that while installation I did mount nfs?
> > I dont understand what else to do next
> >
> >
> >
> >> Hi, even I am facing the same problem.. and here are the logs from
> >> today:
> >>
> >>
> >>
> >> 2017-01-18 12:16:46,100 DEBUG
> >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> (RouterStatusMonitor-1:ctx-bc5cb4cd) (logid:0ffac658) Found 0
> networks
> >> to update RvR status.
> >> 2017-01-18 12:16:46,200 DEBUG [c.c.a.m.AgentManagerImpl]
> >> (AgentManager-Handler-11:null) (logid:) Ping from 2(s-2-VM)
> >> 2017-01-18 12:16:47,521 DEBUG [c.c.a.m.AgentManagerImpl]
> >> (AgentManager-Handler-15:null) (logid:) Ping from 3(v-1-VM)
> >> 2017-01-18 12:16:49,834 DEBUG [c.c.a.m.AgentManagerImpl]
> >> (AgentManager-Handler-9:null) (logid:) SeqA 3-438472: Processing
> Seq
> >> 3-438472: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
> >> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,
> >> "_loadInfo":"{\n
> >> \"connections\": []\n}","wait":0}}] }
> >> 2017-01-18 12:16:49,887 DEBUG [c.c.a.m.AgentManagerImpl]
> >> (AgentManager-Handler-9:null) (logid:) SeqA 3-438472: Sending Seq
> >> 3-438472: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1,
> Flags:
> >> 100010,
> >> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}]
> >> }
> >> 2017-01-18 12:16:54,390 DEBUG [c.c.h.d.HostDaoImpl]
> >> (ClusteredAgentManager
> >> Timer:ctx-8464e4b3) (logid:1f15c721) Resetting hosts suitable for
> >> reconnect
> >> 2017-01-18 12:16:54,392 DEBUG [c.c.h.d.HostDaoImpl]
> >> (ClusteredAgentManager
> >> Timer:ctx-8464e4b3) (logid:1f15c721) Completed resetting hosts
> >> suitable for reconnect
> >> 2017-01-18 12:16:54,392 DEBUG [c.c.h.d.HostDaoImpl]
> >> (ClusteredAgentManager
> >> Timer:ctx-8464e4b3) (logid:1f15c721) Acquiring hosts for clusters
> >> already owned by this management server
> >> 2017-01-18 12:16:54,393 DEBUG [c.c.h.d.HostDaoImpl]
> >> (ClusteredAgentManager
> >> Timer:ctx-8464e4b3) (logid:1f15c721) Completed acquiring hosts for
> >> clusters already owned by this management server
> >> 2017-01-18 12:16:54,393 DEBUG [c.c.h.d.HostDaoImpl]
> >> (ClusteredAgentManager
> >> Timer:ctx-8464e4b3) (logid:1f15c721) Acquiring hosts for clusters
> not
> >> owned by any management server
> >> 2017-01-18 12:16:54,394 DEBUG [c.c.h.d.HostDaoImpl]
> >> (ClusteredAgentManager
> >> Timer:ctx-8464e4b3) (logid:1f15c721) Completed acquiring hosts for
> >> clusters not owned by any management server
> >> 2017-01-18 12:16:54,795 DEBUG [c.c.a.m.AgentManagerImpl]
> >> (AgentManager-Handler-5:null) (logid:) SeqA 3-438473: Processing
> Seq
> >> 3-438473: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
> >> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,
> >> "_loadInfo":"{\n
> >> \"connections\": []\n}","wait":0}}] }
> >> 2017-01-18 12:16:54,854 DEBUG [c.c.a.m.AgentManagerImpl]
> >> (AgentManager-Handler-5:null) (logid:) SeqA 3-438473: Sending Seq
> >> 3-438473: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1,
> Flags:
> >> 100010,
> >> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}]
> >> }
> >> 2017-01-18 12:16:55,984 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> (AsyncJobMgr-Heartbeat-1:ctx-d0469453) (logid:af5d4f47) Begin
> cleanup
> >> expired async-jobs
> >> 2017-01-18 12:16:55,991 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> (AsyncJobMgr-Heartbeat-1:ctx-d0469453) (logid:af5d4f47) End
> cleanup
> >> expired async-jobs
> >> 2017-01-18 12:16:59,245 DEBUG [c.c.s.StatsCollector]
> >> (StatsCollector-1:ctx-68069dff) (logid:a7a6974d) VmStatsCollector
> is
> >> running...
> >> 2017-01-18 12:17:00,841 DEBUG
> [o.a.c.s.SecondaryStorageManagerImpl]
> >> (secstorage-1:ctx-4e101aa5) (logid:65f1f0cb) Zone 1 is ready to
> launch
> >> secondary storage VM
> >> 2017-01-18 12:17:01,095 DEBUG [c.c.c.ConsoleProxyManagerImpl]
> >> (consoleproxy-1:ctx-393828fa) (logid:7b3d0491) Zone 1 is ready to
> >> launch console proxy
> >> 2017-01-18 12:17:03,283 DEBUG [c.c.s.StatsCollector]
> >> (StatsCollector-3:ctx-cc1f58a1) (logid:3ae41340) HostStatsCollector
> is
> >> running...
> >> 2017-01-18 12:17:03,343 DEBUG [c.c.a.t.Request]
> >> (StatsCollector-3:ctx-cc1f58a1) (logid:3ae41340) Seq
> >> 1-6227070909769882804: Received: { Ans: , MgmtId:
> 279278805450954,
> >> via:
> >> 1(iuatc), Ver: v1, Flags: 10, { GetHostStatsAnswer } }
> >> 2017-01-18 12:17:04,796 DEBUG [c.c.a.m.AgentManagerImpl]
> >> (AgentManager-Handler-10:null) (logid:) SeqA 3-438474: Processing
> Seq
> >> 3-438474: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
> >> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,
> >> "_loadInfo":"{\n
> >> \"connections\": []\n}","wait":0}}] }
> >> 2017-01-18 12:17:04,847 DEBUG [c.c.a.m.AgentManagerImpl]
> >> (AgentManager-Handler-10:null) (logid:) SeqA 3-438474: Sending Seq
> >> 3-438474: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1,
> Flags:
> >> 100010,
> >> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}]
> >> }
> >> 2017-01-18 12:17:05,985 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> (AsyncJobMgr-Heartbeat-1:ctx-7f0de3b8) (logid:e9bbd63a) Begin
> cleanup
> >> expired async-jobs
> >> 2017-01-18 12:17:05,991 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> (AsyncJobMgr-Heartbeat-1:ctx-7f0de3b8) (logid:e9bbd63a) End
> cleanup
> >> expired async-jobs
> >> 2017-01-18 12:17:14,798 DEBUG [c.c.a.m.AgentManagerImpl]
> >> (AgentManager-Handler-8:null) (logid:) SeqA 3-438475: Processing
> Seq
> >> 3-438475: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
> >> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,
> >> "_loadInfo":"{\n
> >> \"connections\": []\n}","wait":0}}] }
> >> 2017-01-18 12:17:14,857 DEBUG [c.c.a.m.AgentManagerImpl]
> >> (AgentManager-Handler-8:null) (logid:) SeqA 3-438475: Sending Seq
> >> 3-438475: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1,
> Flags:
> >> 100010,
> >> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}]
> >> }
> >> 2017-01-18 12:17:15,984 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> (AsyncJobMgr-Heartbeat-1:ctx-5682fe3c) (logid:901efb52) Begin
> cleanup
> >> expired async-jobs
> >> 2017-01-18 12:17:15,990 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> (AsyncJobMgr-Heartbeat-1:ctx-5682fe3c) (logid:901efb52) End
> cleanup
> >> expired async-jobs
> >> 2017-01-18 12:17:16,046 DEBUG
> >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> (RouterStatusMonitor-1:ctx-7ad0b18a) (logid:b5b19f2b) Found 0
> routers
> >> to update status.
> >> 2017-01-18 12:17:16,047 DEBUG
> >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> (RouterStatusMonitor-1:ctx-7ad0b18a) (logid:b5b19f2b) Found 0 VPC
> >> networks to update Redundant State.
> >> 2017-01-18 12:17:16,049 DEBUG
> >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> (RouterStatusMonitor-1:ctx-7ad0b18a) (logid:b5b19f2b) Found 0
> networks
> >> to update RvR status.
> >> 2017-01-18 12:17:16,097 DEBUG
> >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> (RouterStatusMonitor-1:ctx-10a8aefd) (logid:b09ffa05) Found 0
> routers
> >> to update status.
> >> 2017-01-18 12:17:16,099 DEBUG
> >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> (RouterStatusMonitor-1:ctx-10a8aefd) (logid:b09ffa05) Found 0 VPC
> >> networks to update Redundant State.
> >> 2017-01-18 12:17:16,101 DEBUG
> >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> (RouterStatusMonitor-1:ctx-10a8aefd) (logid:b09ffa05) Found 0
> networks
> >> to update RvR status.
> >> 2017-01-18 12:17:19,799 DEBUG [c.c.a.m.AgentManagerImpl]
> >> (AgentManager-Handler-3:null) (logid:) SeqA 3-438476: Processing
> Seq
> >> 3-438476: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
> >> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,
> >> "_loadInfo":"{\n
> >> \"connections\": []\n}","wait":0}}] }
> >> 2017-01-18 12:17:19,925 DEBUG [c.c.a.m.AgentManagerImpl]
> >> (AgentManager-Handler-3:null) (logid:) SeqA 3-438476: Sending Seq
> >> 3-438476: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1,
> Flags:
> >> 100010,
> >> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}]
> >> }
> >> 2017-01-18 12:17:25,985 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> (AsyncJobMgr-Heartbeat-1:ctx-a4c8ceb4) (logid:d0d85ed6) Begin
> cleanup
> >> expired async-jobs
> >> 2017-01-18 12:17:25,991 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> (AsyncJobMgr-Heartbeat-1:ctx-a4c8ceb4) (logid:d0d85ed6) End
> cleanup
> >> expired async-jobs
> >> 2017-01-18 12:17:29,800 DEBUG [c.c.a.m.AgentManagerImpl]
> >> (AgentManager-Handler-7:null) (logid:) SeqA 3-438477: Processing
> Seq
> >> 3-438477: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
> >> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,
> >> "_loadInfo":"{\n
> >> \"connections\": []\n}","wait":0}}] }
> >> 2017-01-18 12:17:29,860 DEBUG [c.c.a.m.AgentManagerImpl]
> >> (AgentManager-Handler-7:null) (logid:) SeqA 3-438477: Sending Seq
> >> 3-438477: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1,
> Flags:
> >> 100010,
> >> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}]
> >> }
> >> 2017-01-18 12:17:30,844 DEBUG
> [o.a.c.s.SecondaryStorageManagerImpl]
> >> (secstorage-1:ctx-f2d92c7a) (logid:6bcbea1f) Zone 1 is ready to
> launch
> >> secondary storage VM
> >> 2017-01-18 12:17:31,095 DEBUG [c.c.c.ConsoleProxyManagerImpl]
> >> (consoleproxy-1:ctx-5c5c1698) (logid:a199e87c) Zone 1 is ready to
> >> launch console proxy
> >> 2017-01-18 12:17:34,801 DEBUG [c.c.a.m.AgentManagerImpl]
> >> (AgentManager-Handler-4:null) (logid:) SeqA 3-438478: Processing
> Seq
> >> 3-438478: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
> >> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,
> >> "_loadInfo":"{\n
> >> \"connections\": []\n}","wait":0}}] }
> >> 2017-01-18 12:17:34,853 DEBUG [c.c.a.m.AgentManagerImpl]
> >> (AgentManager-Handler-4:null) (logid:) SeqA 3-438478: Sending Seq
> >> 3-438478: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1,
> Flags:
> >> 100010,
> >> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}]
> >> }
> >> 2017-01-18 12:17:35,534 DEBUG [c.c.s.StatsCollector]
> >> (StatsCollector-4:ctx-37da523c) (logid:aecb957a) AutoScaling
> Monitor
> >> is running...
> >> 2017-01-18 12:17:35,984 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> (AsyncJobMgr-Heartbeat-1:ctx-d031d5da) (logid:722c8ae3) Begin
> cleanup
> >> expired async-jobs
> >> 2017-01-18 12:17:35,990 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> (AsyncJobMgr-Heartbeat-1:ctx-d031d5da) (logid:722c8ae3) End
> cleanup
> >> expired async-jobs
> >> 2017-01-18 12:17:44,802 DEBUG [c.c.a.m.AgentManagerImpl]
> >> (AgentManager-Handler-1:null) (logid:) SeqA 3-438479: Processing
> Seq
> >> 3-438479: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
> >> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,
> >> "_loadInfo":"{\n
> >> \"connections\": []\n}","wait":0}}] }
> >> 2017-01-18 12:17:44,863 DEBUG [c.c.a.m.AgentManagerImpl]
> >> (AgentManager-Handler-1:null) (logid:) SeqA 3-438479: Sending Seq
> >> 3-438479: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1,
> Flags:
> >> 100010,
> >> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}]
> >> }
> >> 2017-01-18 12:17:45,754 DEBUG [c.c.a.m.AgentManagerImpl]
> >> (AgentManager-Handler-12:null) (logid:) Ping from 1(iuatc)
> >> 2017-01-18 12:17:45,754 DEBUG
> [c.c.v.VirtualMachinePowerStateSyncImpl]
> >> (AgentManager-Handler-12:null) (logid:) Process host VM state
> report
> >> from ping process. host: 1
> >> 2017-01-18 12:17:45,760 DEBUG
> [c.c.v.VirtualMachinePowerStateSyncImpl]
> >> (AgentManager-Handler-12:null) (logid:) Process VM state report.
> host:
> >> 1, number of records in report: 2
> >> 2017-01-18 12:17:45,760 DEBUG
> [c.c.v.VirtualMachinePowerStateSyncImpl]
> >> (AgentManager-Handler-12:null) (logid:) VM state report. host: 1,
> vm
> >> id:
> >> 1, power state: PowerOn
> >> 2017-01-18 12:17:45,763 DEBUG
> [c.c.v.VirtualMachinePowerStateSyncImpl]
> >> (AgentManager-Handler-12:null) (logid:) VM power state does not
> >> change, skip DB writing. vm id: 1
> >> 2017-01-18 12:17:45,763 DEBUG
> [c.c.v.VirtualMachinePowerStateSyncImpl]
> >> (AgentManager-Handler-12:null) (logid:) VM state report. host: 1,
> vm
> >> id:
> >> 2, power state: PowerOn
> >> 2017-01-18 12:17:45,766 DEBUG
> [c.c.v.VirtualMachinePowerStateSyncImpl]
> >> (AgentManager-Handler-12:null) (logid:) VM power state does not
> >> change, skip DB writing. vm id: 2
> >> 2017-01-18 12:17:45,769 DEBUG
> [c.c.v.VirtualMachinePowerStateSyncImpl]
> >> (AgentManager-Handler-12:null) (logid:) Done with process of VM
> state
> >> report. host: 1
> >> 2017-01-18 12:17:45,985 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> (AsyncJobMgr-Heartbeat-1:ctx-b39e7e84) (logid:d0f90660) Begin
> cleanup
> >> expired async-jobs
> >> 2017-01-18 12:17:45,991 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> (AsyncJobMgr-Heartbeat-1:ctx-b39e7e84) (logid:d0f90660) End
> cleanup
> >> expired async-jobs
> >> 2017-01-18 12:17:46,046 DEBUG
> >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> (RouterStatusMonitor-1:ctx-36fbb5c2) (logid:d346d004) Found 0
> routers
> >> to update status.
> >> 2017-01-18 12:17:46,047 DEBUG
> >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> (RouterStatusMonitor-1:ctx-36fbb5c2) (logid:d346d004) Found 0 VPC
> >> networks to update Redundant State.
> >> 2017-01-18 12:17:46,049 DEBUG
> >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> (RouterStatusMonitor-1:ctx-36fbb5c2) (logid:d346d004) Found 0
> networks
> >> to update RvR status.
> >> 2017-01-18 12:17:46,097 DEBUG
> >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> (RouterStatusMonitor-1:ctx-16234697) (logid:beb5fec7) Found 0
> routers
> >> to update status.
> >> 2017-01-18 12:17:46,099 DEBUG
> >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> (RouterStatusMonitor-1:ctx-16234697) (logid:beb5fec7) Found 0 VPC
> >> networks to update Redundant State.
> >> 2017-01-18 12:17:46,100 DEBUG
> >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> (RouterStatusMonitor-1:ctx-16234697) (logid:beb5fec7) Found 0
> networks
> >> to update RvR status.
> >> 2017-01-18 12:17:46,201 DEBUG [c.c.a.m.AgentManagerImpl]
> >> (AgentManager-Handler-2:null) (logid:) Ping from 2(s-2-VM)
> >> 2017-01-18 12:17:47,521 DEBUG [c.c.a.m.AgentManagerImpl]
> >> (AgentManager-Handler-14:null) (logid:) Ping from 3(v-1-VM)
> >> 2017-01-18 12:17:49,842 DEBUG [c.c.a.m.AgentManagerImpl]
> >> (AgentManager-Handler-13:null) (logid:) SeqA 3-438481: Processing
> Seq
> >> 3-438481: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
> >> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,
> >> "_loadInfo":"{\n
> >> \"connections\": []\n}","wait":0}}] }
> >> 2017-01-18 12:17:49,897 DEBUG [c.c.a.m.AgentManagerImpl]
> >> (AgentManager-Handler-13:null) (logid:) SeqA 3-438481: Sending Seq
> >> 3-438481: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1,
> Flags:
> >> 100010,
> >> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}]
> >> }
> >> 2017-01-18 12:17:55,985 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> (AsyncJobMgr-Heartbeat-1:ctx-c0db35b5) (logid:c304f172) Begin
> cleanup
> >> expired async-jobs
> >> 2017-01-18 12:17:55,991 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> (AsyncJobMgr-Heartbeat-1:ctx-c0db35b5) (logid:c304f172) End
> cleanup
> >> expired async-jobs
> >> 2017-01-18 12:17:59,250 DEBUG [c.c.s.StatsCollector]
> >> (StatsCollector-6:ctx-eec84206) (logid:b88e0ce8) VmStatsCollector
> is
> >> running...
> >> 2017-01-18 12:17:59,803 DEBUG [c.c.a.m.AgentManagerImpl]
> >> (AgentManager-Handler-11:null) (logid:) SeqA 3-438482: Processing
> Seq
> >> 3-438482: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
> >> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,
> >> "_loadInfo":"{\n
> >> \"connections\": []\n}","wait":0}}] }
> >> 2017-01-18 12:17:59,857 DEBUG [c.c.a.m.AgentManagerImpl]
> >> (AgentManager-Handler-11:null) (logid:) SeqA 3-438482: Sending Seq
> >> 3-438482: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1,
> Flags:
> >> 100010,
> >> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}]
> >> }
> >> 2017-01-18 12:18:00,843 DEBUG
> [o.a.c.s.SecondaryStorageManagerImpl]
> >> (secstorage-1:ctx-f45be089) (logid:934bd9f7) Zone 1 is ready to
> launch
> >> secondary storage VM
> >> 2017-01-18 12:18:01,094 DEBUG [c.c.c.ConsoleProxyManagerImpl]
> >> (consoleproxy-1:ctx-4b11dfe0) (logid:cce52dbb) Zone 1 is ready to
> >> launch console proxy
> >> 2017-01-18 12:18:03,345 DEBUG [c.c.s.StatsCollector]
> >> (StatsCollector-2:ctx-c743d4ee) (logid:c1f62e31) HostStatsCollector
> is
> >> running...
> >> 2017-01-18 12:18:03,406 DEBUG [c.c.a.t.Request]
> >> (StatsCollector-2:ctx-c743d4ee) (logid:c1f62e31) Seq
> >> 1-6227070909769882805: Received: { Ans: , MgmtId:
> 279278805450954,
> >> via:
> >> 1(iuatc), Ver: v1, Flags: 10, { GetHostStatsAnswer } }
> >> 2017-01-18 12:18:04,804 DEBUG [c.c.a.m.AgentManagerImpl]
> >> (AgentManager-Handler-9:null) (logid:) SeqA 3-438483: Processing
> Seq
> >> 3-438483: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
> >> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,
> >> "_loadInfo":"{\n
> >> \"connections\": []\n}","wait":0}}] }
> >> 2017-01-18 12:18:04,858 DEBUG [c.c.a.m.AgentManagerImpl]
> >> (AgentManager-Handler-9:null) (logid:) SeqA 3-438483: Sending Seq
> >> 3-438483: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1,
> Flags:
> >> 100010,
> >> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}]
> >> }
> >> 2017-01-18 12:18:05,985 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> (AsyncJobMgr-Heartbeat-1:ctx-149380f2) (logid:53d166f9) Begin
> cleanup
> >> expired async-jobs
> >> 2017-01-18 12:18:05,990 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> (AsyncJobMgr-Heartbeat-1:ctx-149380f2) (logid:53d166f9) End
> cleanup
> >> expired async-jobs
> >> 2017-01-18 12:18:14,805 DEBUG [c.c.a.m.AgentManagerImpl]
> >> (AgentManager-Handler-5:null) (logid:) SeqA 3-438484: Processing
> Seq
> >> 3-438484: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
> >> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,
> >> "_loadInfo":"{\n
> >> \"connections\": []\n}","wait":0}}] }
> >> 2017-01-18 12:18:14,951 DEBUG [c.c.a.m.AgentManagerImpl]
> >> (AgentManager-Handler-5:null) (logid:) SeqA 3-438484: Sending Seq
> >> 3-438484: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1,
> Flags:
> >> 100010,
> >> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}]
> >> }
> >> 2017-01-18 12:18:15,985 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> (AsyncJobMgr-Heartbeat-1:ctx-7ccabf83) (logid:2b0f34c4) Begin
> cleanup
> >> expired async-jobs
> >> 2017-01-18 12:18:15,993 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> (AsyncJobMgr-Heartbeat-1:ctx-7ccabf83) (logid:2b0f34c4) End
> cleanup
> >> expired async-jobs
> >> 2017-01-18 12:18:16,046 DEBUG
> >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> (RouterStatusMonitor-1:ctx-fc5408a0) (logid:bbd625c1) Found 0
> routers
> >> to update status.
> >> 2017-01-18 12:18:16,048 DEBUG
> >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> (RouterStatusMonitor-1:ctx-fc5408a0) (logid:bbd625c1) Found 0 VPC
> >> networks to update Redundant State.
> >> 2017-01-18 12:18:16,049 DEBUG
> >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> (RouterStatusMonitor-1:ctx-fc5408a0) (logid:bbd625c1) Found 0
> networks
> >> to update RvR status.
> >> 2017-01-18 12:18:16,097 DEBUG
> >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> (RouterStatusMonitor-1:ctx-2c697f6b) (logid:cfa955f2) Found 0
> routers
> >> to update status.
> >> 2017-01-18 12:18:16,099 DEBUG
> >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> (RouterStatusMonitor-1:ctx-2c697f6b) (logid:cfa955f2) Found 0 VPC
> >> networks to update Redundant State.
> >> 2017-01-18 12:18:16,101 DEBUG
> >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> (RouterStatusMonitor-1:ctx-2c697f6b) (logid:cfa955f2) Found 0
> networks
> >> to update RvR status.
> >> 2017-01-18 12:18:24,389 DEBUG [c.c.h.d.HostDaoImpl]
> >> (ClusteredAgentManager
> >> Timer:ctx-d699e7ed) (logid:e10673e8) Resetting hosts suitable for
> >> reconnect
> >> 2017-01-18 12:18:24,392 DEBUG [c.c.h.d.HostDaoImpl]
> >> (ClusteredAgentManager
> >> Timer:ctx-d699e7ed) (logid:e10673e8) Completed resetting hosts
> >> suitable for reconnect
> >> 2017-01-18 12:18:24,392 DEBUG [c.c.h.d.HostDaoImpl]
> >> (ClusteredAgentManager
> >> Timer:ctx-d699e7ed) (logid:e10673e8) Acquiring hosts for clusters
> >> already owned by this management server
> >> 2017-01-18 12:18:24,392 DEBUG [c.c.h.d.HostDaoImpl]
> >> (ClusteredAgentManager
> >> Timer:ctx-d699e7ed) (logid:e10673e8) Completed acquiring hosts for
> >> clusters already owned by this management server
> >> 2017-01-18 12:18:24,392 DEBUG [c.c.h.d.HostDaoImpl]
> >> (ClusteredAgentManager
> >> Timer:ctx-d699e7ed) (logid:e10673e8) Acquiring hosts for clusters
> not
> >> owned by any management server
> >> 2017-01-18 12:18:24,393 DEBUG [c.c.h.d.HostDaoImpl]
> >> (ClusteredAgentManager
> >> Timer:ctx-d699e7ed) (logid:e10673e8) Completed acquiring hosts for
> >> clusters not owned by any management server
> >> 2017-01-18 12:18:24,807 DEBUG [c.c.a.m.AgentManagerImpl]
> >> (AgentManager-Handler-6:null) (logid:) SeqA 3-438485: Processing
> Seq
> >> 3-438485: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
> >> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,
> >> "_loadInfo":"{\n
> >> \"connections\": []\n}","wait":0}}] }
> >> 2017-01-18 12:18:24,870 DEBUG [c.c.a.m.AgentManagerImpl]
> >> (AgentManager-Handler-6:null) (logid:) SeqA 3-438485: Sending Seq
> >> 3-438485: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1,
> Flags:
> >> 100010, [{"com.cloud.agent.api.AgentControlAnswer":{"result":tru
> >>
> >>> Can you please share your management_server logs
> >>>
> >>> On Tue, Jan 17, 2017 at 11:47 PM,
> ì¡°ëÅ\u2019â\u201a¬ÃË\u0153â\u20ac¢
> >>> <ca...@renet.kr> wrote:
> >>>
> >>>> Hi, all
> >>>>
> >>>> I am facing secondary storage problem.
> >>>>
> >>>> I am using NFS Server to use primary and secondary storage.
> >>>>
> >>>> After rebooting the host,
> >>>>
> >>>> In Dashboard Cloudstack , it says Secondary Storage 0.00 KB /
> 0.00
> >>>> KB
> >>>>
> >>>> Primary Storage is okay.
> >>>>
> >>>>
> >>>>
> >>>> After all, system VMs are not up. It was working fine before
> >>>> rebooting.
> >>>>
> >>>>
> >>>>
> >>>> I have tried several times of restart of Management service.
> >>>>
> >>>>
> >>>>
> >>>> Any ideas?
> >>>>
> >>>>
> >>>>
> >>>> Thanks,
> >>>>
> >>>>
> >>>
> >>
> >>
> >>
> >
> >
> >
>
>
>
>
>
> Dag.Sonstebo@shapeblue.com
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London WC2N 4HSUK
> @shapeblue
>
>
>
>
Re: Cloudstack 4.7 Secondary Storage problem
Posted by Dag Sonstebo <Da...@shapeblue.com>.
Hi Merlin,
What Paul means is you need to log on to the SSVM, then do something like this:
mkdir /tmp/secondary
mount –t NFS <NFS server IP>:/<NFS/path/here>/ /tmp/secondary
This will tell you if your NFS permissions are configured correctly.
Regards,
Dag Sonstebo
Cloud Architect
ShapeBlue
On 18/01/2017, 09:29, "merlin@students.iitmandi.ac.in" <me...@students.iitmandi.ac.in> wrote:
Also when you say mount from ssvm. Does it mean this?
Log into ssvm and run "mount":- the reply is down below. I am not able to
understand much even from the documentation,
I am sorry for the lack of full knowledge in this concern.
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
udev on /dev type devtmpfs (rw,relatime,size=10240k,nr_inodes=62128,mode=755)
devpts on /dev/pts type devpts
(rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,noexec,relatime,size=50896k,mode=755)
/dev/disk/by-uuid/30c81d3d-ee9f-4a88-81c1-5f349b22ba1d on / type ext4
(rw,relatime,errors=remount-ro,user_xattr,barrier=1,data=ordered)
tmpfs on /run/lock type tmpfs (rw,nosuid,nodev,noexec,relatime,size=5120k)
tmpfs on /run/shm type tmpfs (rw,nosuid,nodev,noexec,relatime,size=211960k)
/dev/vda1 on /boot type ext4 (rw,relatime,user_xattr,barrier=1,data=ordered)
/dev/vda6 on /home type ext4 (rw,relatime,user_xattr,barrier=1,data=ordered)
/dev/vda8 on /opt type ext4 (rw,relatime,user_xattr,barrier=1,data=ordered)
/dev/vda11 on /tmp type ext4 (rw,relatime,user_xattr,barrier=1,data=ordered)
/dev/vda7 on /usr type ext4 (rw,relatime,user_xattr,barrier=1,data=ordered)
/dev/vda9 on /var type ext4 (rw,relatime,user_xattr,barrier=1,data=ordered)
/dev/vda10 on /var/log type ext4
(rw,relatime,user_xattr,barrier=1,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
> Hi Merlin,
>
> To troubleshoot, try mounting the NFS yourself from the SSVM and get that
> working.
>
> the likely problem is permissions on the NFS mounts. Either the IP of the
> SSVM is not allowed or the permissions don't include no_root_squash (or
> equivalent) the SSVM is probably connecting as 'cloud', so connections
> have to be mapped to 'root'. Finally the SSVM only reliably supports
> NFSv3
>
>
>
>
> Kind regards,
>
> Paul Angus
>
> paul.angus@shapeblue.comÂ
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London WC2N 4HSUK
> @shapeblue
>
>
>
>
> -----Original Message-----
> From: merlin@students.iitmandi.ac.in
> [mailto:merlin@students.iitmandi.ac.in]
> Sent: 18 January 2017 08:48
> To: users@cloudstack.apache.org
> Subject: Re: Cloudstack 4.7 Secondary Storage problem
>
> And this is the result of check.sh
>
> 2 packets transmitted, 2 packets received, 0% packet loss round-trip
> min/avg/max/stddev = 0.400/0.550/0.701/0.151 ms
> Good: Can ping DNS server
> ================================================
> Good: DNS resolves download.cloud.com
> ================================================
> ERROR: NFS is not currently mounted
> Try manually mounting from inside the VM NFS server is 10.8.0.1 PING
> 10.8.0.1 (10.8.0.1): 48 data bytes
> 56 bytes from 10.8.0.1: icmp_seq=0 ttl=64 time=0.456 ms
> 56 bytes from 10.8.0.1: icmp_seq=1 ttl=64 time=0.708 ms
> --- 10.8.0.1 ping statistics ---
> 2 packets transmitted, 2 packets received, 0% packet loss round-trip
> min/avg/max/stddev = 0.456/0.582/0.708/0.126 ms
> Good: Can ping nfs server
> ================================================
> Management server is 10.8.19.111. Checking connectivity.
> Good: Can connect to management server port 8250
> ================================================
> Good: Java process is running
> ================================================
> Tests Complete. Look for ERROR or WARNING above.
>
>
> The problem is that while installation I did mount nfs?
> I dont understand what else to do next
>
>
>
>> Hi, even I am facing the same problem.. and here are the logs from
>> today:
>>
>>
>>
>> 2017-01-18 12:16:46,100 DEBUG
>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> (RouterStatusMonitor-1:ctx-bc5cb4cd) (logid:0ffac658) Found 0 networks
>> to update RvR status.
>> 2017-01-18 12:16:46,200 DEBUG [c.c.a.m.AgentManagerImpl]
>> (AgentManager-Handler-11:null) (logid:) Ping from 2(s-2-VM)
>> 2017-01-18 12:16:47,521 DEBUG [c.c.a.m.AgentManagerImpl]
>> (AgentManager-Handler-15:null) (logid:) Ping from 3(v-1-VM)
>> 2017-01-18 12:16:49,834 DEBUG [c.c.a.m.AgentManagerImpl]
>> (AgentManager-Handler-9:null) (logid:) SeqA 3-438472: Processing Seq
>> 3-438472: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
>> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,
>> "_loadInfo":"{\n
>> \"connections\": []\n}","wait":0}}] }
>> 2017-01-18 12:16:49,887 DEBUG [c.c.a.m.AgentManagerImpl]
>> (AgentManager-Handler-9:null) (logid:) SeqA 3-438472: Sending Seq
>> 3-438472: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1, Flags:
>> 100010,
>> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}]
>> }
>> 2017-01-18 12:16:54,390 DEBUG [c.c.h.d.HostDaoImpl]
>> (ClusteredAgentManager
>> Timer:ctx-8464e4b3) (logid:1f15c721) Resetting hosts suitable for
>> reconnect
>> 2017-01-18 12:16:54,392 DEBUG [c.c.h.d.HostDaoImpl]
>> (ClusteredAgentManager
>> Timer:ctx-8464e4b3) (logid:1f15c721) Completed resetting hosts
>> suitable for reconnect
>> 2017-01-18 12:16:54,392 DEBUG [c.c.h.d.HostDaoImpl]
>> (ClusteredAgentManager
>> Timer:ctx-8464e4b3) (logid:1f15c721) Acquiring hosts for clusters
>> already owned by this management server
>> 2017-01-18 12:16:54,393 DEBUG [c.c.h.d.HostDaoImpl]
>> (ClusteredAgentManager
>> Timer:ctx-8464e4b3) (logid:1f15c721) Completed acquiring hosts for
>> clusters already owned by this management server
>> 2017-01-18 12:16:54,393 DEBUG [c.c.h.d.HostDaoImpl]
>> (ClusteredAgentManager
>> Timer:ctx-8464e4b3) (logid:1f15c721) Acquiring hosts for clusters not
>> owned by any management server
>> 2017-01-18 12:16:54,394 DEBUG [c.c.h.d.HostDaoImpl]
>> (ClusteredAgentManager
>> Timer:ctx-8464e4b3) (logid:1f15c721) Completed acquiring hosts for
>> clusters not owned by any management server
>> 2017-01-18 12:16:54,795 DEBUG [c.c.a.m.AgentManagerImpl]
>> (AgentManager-Handler-5:null) (logid:) SeqA 3-438473: Processing Seq
>> 3-438473: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
>> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,
>> "_loadInfo":"{\n
>> \"connections\": []\n}","wait":0}}] }
>> 2017-01-18 12:16:54,854 DEBUG [c.c.a.m.AgentManagerImpl]
>> (AgentManager-Handler-5:null) (logid:) SeqA 3-438473: Sending Seq
>> 3-438473: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1, Flags:
>> 100010,
>> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}]
>> }
>> 2017-01-18 12:16:55,984 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-d0469453) (logid:af5d4f47) Begin cleanup
>> expired async-jobs
>> 2017-01-18 12:16:55,991 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-d0469453) (logid:af5d4f47) End cleanup
>> expired async-jobs
>> 2017-01-18 12:16:59,245 DEBUG [c.c.s.StatsCollector]
>> (StatsCollector-1:ctx-68069dff) (logid:a7a6974d) VmStatsCollector is
>> running...
>> 2017-01-18 12:17:00,841 DEBUG [o.a.c.s.SecondaryStorageManagerImpl]
>> (secstorage-1:ctx-4e101aa5) (logid:65f1f0cb) Zone 1 is ready to launch
>> secondary storage VM
>> 2017-01-18 12:17:01,095 DEBUG [c.c.c.ConsoleProxyManagerImpl]
>> (consoleproxy-1:ctx-393828fa) (logid:7b3d0491) Zone 1 is ready to
>> launch console proxy
>> 2017-01-18 12:17:03,283 DEBUG [c.c.s.StatsCollector]
>> (StatsCollector-3:ctx-cc1f58a1) (logid:3ae41340) HostStatsCollector is
>> running...
>> 2017-01-18 12:17:03,343 DEBUG [c.c.a.t.Request]
>> (StatsCollector-3:ctx-cc1f58a1) (logid:3ae41340) Seq
>> 1-6227070909769882804: Received: { Ans: , MgmtId: 279278805450954,
>> via:
>> 1(iuatc), Ver: v1, Flags: 10, { GetHostStatsAnswer } }
>> 2017-01-18 12:17:04,796 DEBUG [c.c.a.m.AgentManagerImpl]
>> (AgentManager-Handler-10:null) (logid:) SeqA 3-438474: Processing Seq
>> 3-438474: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
>> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,
>> "_loadInfo":"{\n
>> \"connections\": []\n}","wait":0}}] }
>> 2017-01-18 12:17:04,847 DEBUG [c.c.a.m.AgentManagerImpl]
>> (AgentManager-Handler-10:null) (logid:) SeqA 3-438474: Sending Seq
>> 3-438474: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1, Flags:
>> 100010,
>> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}]
>> }
>> 2017-01-18 12:17:05,985 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-7f0de3b8) (logid:e9bbd63a) Begin cleanup
>> expired async-jobs
>> 2017-01-18 12:17:05,991 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-7f0de3b8) (logid:e9bbd63a) End cleanup
>> expired async-jobs
>> 2017-01-18 12:17:14,798 DEBUG [c.c.a.m.AgentManagerImpl]
>> (AgentManager-Handler-8:null) (logid:) SeqA 3-438475: Processing Seq
>> 3-438475: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
>> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,
>> "_loadInfo":"{\n
>> \"connections\": []\n}","wait":0}}] }
>> 2017-01-18 12:17:14,857 DEBUG [c.c.a.m.AgentManagerImpl]
>> (AgentManager-Handler-8:null) (logid:) SeqA 3-438475: Sending Seq
>> 3-438475: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1, Flags:
>> 100010,
>> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}]
>> }
>> 2017-01-18 12:17:15,984 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-5682fe3c) (logid:901efb52) Begin cleanup
>> expired async-jobs
>> 2017-01-18 12:17:15,990 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-5682fe3c) (logid:901efb52) End cleanup
>> expired async-jobs
>> 2017-01-18 12:17:16,046 DEBUG
>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> (RouterStatusMonitor-1:ctx-7ad0b18a) (logid:b5b19f2b) Found 0 routers
>> to update status.
>> 2017-01-18 12:17:16,047 DEBUG
>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> (RouterStatusMonitor-1:ctx-7ad0b18a) (logid:b5b19f2b) Found 0 VPC
>> networks to update Redundant State.
>> 2017-01-18 12:17:16,049 DEBUG
>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> (RouterStatusMonitor-1:ctx-7ad0b18a) (logid:b5b19f2b) Found 0 networks
>> to update RvR status.
>> 2017-01-18 12:17:16,097 DEBUG
>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> (RouterStatusMonitor-1:ctx-10a8aefd) (logid:b09ffa05) Found 0 routers
>> to update status.
>> 2017-01-18 12:17:16,099 DEBUG
>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> (RouterStatusMonitor-1:ctx-10a8aefd) (logid:b09ffa05) Found 0 VPC
>> networks to update Redundant State.
>> 2017-01-18 12:17:16,101 DEBUG
>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> (RouterStatusMonitor-1:ctx-10a8aefd) (logid:b09ffa05) Found 0 networks
>> to update RvR status.
>> 2017-01-18 12:17:19,799 DEBUG [c.c.a.m.AgentManagerImpl]
>> (AgentManager-Handler-3:null) (logid:) SeqA 3-438476: Processing Seq
>> 3-438476: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
>> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,
>> "_loadInfo":"{\n
>> \"connections\": []\n}","wait":0}}] }
>> 2017-01-18 12:17:19,925 DEBUG [c.c.a.m.AgentManagerImpl]
>> (AgentManager-Handler-3:null) (logid:) SeqA 3-438476: Sending Seq
>> 3-438476: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1, Flags:
>> 100010,
>> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}]
>> }
>> 2017-01-18 12:17:25,985 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-a4c8ceb4) (logid:d0d85ed6) Begin cleanup
>> expired async-jobs
>> 2017-01-18 12:17:25,991 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-a4c8ceb4) (logid:d0d85ed6) End cleanup
>> expired async-jobs
>> 2017-01-18 12:17:29,800 DEBUG [c.c.a.m.AgentManagerImpl]
>> (AgentManager-Handler-7:null) (logid:) SeqA 3-438477: Processing Seq
>> 3-438477: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
>> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,
>> "_loadInfo":"{\n
>> \"connections\": []\n}","wait":0}}] }
>> 2017-01-18 12:17:29,860 DEBUG [c.c.a.m.AgentManagerImpl]
>> (AgentManager-Handler-7:null) (logid:) SeqA 3-438477: Sending Seq
>> 3-438477: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1, Flags:
>> 100010,
>> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}]
>> }
>> 2017-01-18 12:17:30,844 DEBUG [o.a.c.s.SecondaryStorageManagerImpl]
>> (secstorage-1:ctx-f2d92c7a) (logid:6bcbea1f) Zone 1 is ready to launch
>> secondary storage VM
>> 2017-01-18 12:17:31,095 DEBUG [c.c.c.ConsoleProxyManagerImpl]
>> (consoleproxy-1:ctx-5c5c1698) (logid:a199e87c) Zone 1 is ready to
>> launch console proxy
>> 2017-01-18 12:17:34,801 DEBUG [c.c.a.m.AgentManagerImpl]
>> (AgentManager-Handler-4:null) (logid:) SeqA 3-438478: Processing Seq
>> 3-438478: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
>> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,
>> "_loadInfo":"{\n
>> \"connections\": []\n}","wait":0}}] }
>> 2017-01-18 12:17:34,853 DEBUG [c.c.a.m.AgentManagerImpl]
>> (AgentManager-Handler-4:null) (logid:) SeqA 3-438478: Sending Seq
>> 3-438478: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1, Flags:
>> 100010,
>> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}]
>> }
>> 2017-01-18 12:17:35,534 DEBUG [c.c.s.StatsCollector]
>> (StatsCollector-4:ctx-37da523c) (logid:aecb957a) AutoScaling Monitor
>> is running...
>> 2017-01-18 12:17:35,984 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-d031d5da) (logid:722c8ae3) Begin cleanup
>> expired async-jobs
>> 2017-01-18 12:17:35,990 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-d031d5da) (logid:722c8ae3) End cleanup
>> expired async-jobs
>> 2017-01-18 12:17:44,802 DEBUG [c.c.a.m.AgentManagerImpl]
>> (AgentManager-Handler-1:null) (logid:) SeqA 3-438479: Processing Seq
>> 3-438479: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
>> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,
>> "_loadInfo":"{\n
>> \"connections\": []\n}","wait":0}}] }
>> 2017-01-18 12:17:44,863 DEBUG [c.c.a.m.AgentManagerImpl]
>> (AgentManager-Handler-1:null) (logid:) SeqA 3-438479: Sending Seq
>> 3-438479: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1, Flags:
>> 100010,
>> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}]
>> }
>> 2017-01-18 12:17:45,754 DEBUG [c.c.a.m.AgentManagerImpl]
>> (AgentManager-Handler-12:null) (logid:) Ping from 1(iuatc)
>> 2017-01-18 12:17:45,754 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
>> (AgentManager-Handler-12:null) (logid:) Process host VM state report
>> from ping process. host: 1
>> 2017-01-18 12:17:45,760 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
>> (AgentManager-Handler-12:null) (logid:) Process VM state report. host:
>> 1, number of records in report: 2
>> 2017-01-18 12:17:45,760 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
>> (AgentManager-Handler-12:null) (logid:) VM state report. host: 1, vm
>> id:
>> 1, power state: PowerOn
>> 2017-01-18 12:17:45,763 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
>> (AgentManager-Handler-12:null) (logid:) VM power state does not
>> change, skip DB writing. vm id: 1
>> 2017-01-18 12:17:45,763 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
>> (AgentManager-Handler-12:null) (logid:) VM state report. host: 1, vm
>> id:
>> 2, power state: PowerOn
>> 2017-01-18 12:17:45,766 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
>> (AgentManager-Handler-12:null) (logid:) VM power state does not
>> change, skip DB writing. vm id: 2
>> 2017-01-18 12:17:45,769 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
>> (AgentManager-Handler-12:null) (logid:) Done with process of VM state
>> report. host: 1
>> 2017-01-18 12:17:45,985 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-b39e7e84) (logid:d0f90660) Begin cleanup
>> expired async-jobs
>> 2017-01-18 12:17:45,991 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-b39e7e84) (logid:d0f90660) End cleanup
>> expired async-jobs
>> 2017-01-18 12:17:46,046 DEBUG
>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> (RouterStatusMonitor-1:ctx-36fbb5c2) (logid:d346d004) Found 0 routers
>> to update status.
>> 2017-01-18 12:17:46,047 DEBUG
>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> (RouterStatusMonitor-1:ctx-36fbb5c2) (logid:d346d004) Found 0 VPC
>> networks to update Redundant State.
>> 2017-01-18 12:17:46,049 DEBUG
>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> (RouterStatusMonitor-1:ctx-36fbb5c2) (logid:d346d004) Found 0 networks
>> to update RvR status.
>> 2017-01-18 12:17:46,097 DEBUG
>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> (RouterStatusMonitor-1:ctx-16234697) (logid:beb5fec7) Found 0 routers
>> to update status.
>> 2017-01-18 12:17:46,099 DEBUG
>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> (RouterStatusMonitor-1:ctx-16234697) (logid:beb5fec7) Found 0 VPC
>> networks to update Redundant State.
>> 2017-01-18 12:17:46,100 DEBUG
>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> (RouterStatusMonitor-1:ctx-16234697) (logid:beb5fec7) Found 0 networks
>> to update RvR status.
>> 2017-01-18 12:17:46,201 DEBUG [c.c.a.m.AgentManagerImpl]
>> (AgentManager-Handler-2:null) (logid:) Ping from 2(s-2-VM)
>> 2017-01-18 12:17:47,521 DEBUG [c.c.a.m.AgentManagerImpl]
>> (AgentManager-Handler-14:null) (logid:) Ping from 3(v-1-VM)
>> 2017-01-18 12:17:49,842 DEBUG [c.c.a.m.AgentManagerImpl]
>> (AgentManager-Handler-13:null) (logid:) SeqA 3-438481: Processing Seq
>> 3-438481: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
>> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,
>> "_loadInfo":"{\n
>> \"connections\": []\n}","wait":0}}] }
>> 2017-01-18 12:17:49,897 DEBUG [c.c.a.m.AgentManagerImpl]
>> (AgentManager-Handler-13:null) (logid:) SeqA 3-438481: Sending Seq
>> 3-438481: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1, Flags:
>> 100010,
>> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}]
>> }
>> 2017-01-18 12:17:55,985 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-c0db35b5) (logid:c304f172) Begin cleanup
>> expired async-jobs
>> 2017-01-18 12:17:55,991 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-c0db35b5) (logid:c304f172) End cleanup
>> expired async-jobs
>> 2017-01-18 12:17:59,250 DEBUG [c.c.s.StatsCollector]
>> (StatsCollector-6:ctx-eec84206) (logid:b88e0ce8) VmStatsCollector is
>> running...
>> 2017-01-18 12:17:59,803 DEBUG [c.c.a.m.AgentManagerImpl]
>> (AgentManager-Handler-11:null) (logid:) SeqA 3-438482: Processing Seq
>> 3-438482: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
>> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,
>> "_loadInfo":"{\n
>> \"connections\": []\n}","wait":0}}] }
>> 2017-01-18 12:17:59,857 DEBUG [c.c.a.m.AgentManagerImpl]
>> (AgentManager-Handler-11:null) (logid:) SeqA 3-438482: Sending Seq
>> 3-438482: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1, Flags:
>> 100010,
>> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}]
>> }
>> 2017-01-18 12:18:00,843 DEBUG [o.a.c.s.SecondaryStorageManagerImpl]
>> (secstorage-1:ctx-f45be089) (logid:934bd9f7) Zone 1 is ready to launch
>> secondary storage VM
>> 2017-01-18 12:18:01,094 DEBUG [c.c.c.ConsoleProxyManagerImpl]
>> (consoleproxy-1:ctx-4b11dfe0) (logid:cce52dbb) Zone 1 is ready to
>> launch console proxy
>> 2017-01-18 12:18:03,345 DEBUG [c.c.s.StatsCollector]
>> (StatsCollector-2:ctx-c743d4ee) (logid:c1f62e31) HostStatsCollector is
>> running...
>> 2017-01-18 12:18:03,406 DEBUG [c.c.a.t.Request]
>> (StatsCollector-2:ctx-c743d4ee) (logid:c1f62e31) Seq
>> 1-6227070909769882805: Received: { Ans: , MgmtId: 279278805450954,
>> via:
>> 1(iuatc), Ver: v1, Flags: 10, { GetHostStatsAnswer } }
>> 2017-01-18 12:18:04,804 DEBUG [c.c.a.m.AgentManagerImpl]
>> (AgentManager-Handler-9:null) (logid:) SeqA 3-438483: Processing Seq
>> 3-438483: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
>> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,
>> "_loadInfo":"{\n
>> \"connections\": []\n}","wait":0}}] }
>> 2017-01-18 12:18:04,858 DEBUG [c.c.a.m.AgentManagerImpl]
>> (AgentManager-Handler-9:null) (logid:) SeqA 3-438483: Sending Seq
>> 3-438483: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1, Flags:
>> 100010,
>> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}]
>> }
>> 2017-01-18 12:18:05,985 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-149380f2) (logid:53d166f9) Begin cleanup
>> expired async-jobs
>> 2017-01-18 12:18:05,990 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-149380f2) (logid:53d166f9) End cleanup
>> expired async-jobs
>> 2017-01-18 12:18:14,805 DEBUG [c.c.a.m.AgentManagerImpl]
>> (AgentManager-Handler-5:null) (logid:) SeqA 3-438484: Processing Seq
>> 3-438484: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
>> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,
>> "_loadInfo":"{\n
>> \"connections\": []\n}","wait":0}}] }
>> 2017-01-18 12:18:14,951 DEBUG [c.c.a.m.AgentManagerImpl]
>> (AgentManager-Handler-5:null) (logid:) SeqA 3-438484: Sending Seq
>> 3-438484: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1, Flags:
>> 100010,
>> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}]
>> }
>> 2017-01-18 12:18:15,985 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-7ccabf83) (logid:2b0f34c4) Begin cleanup
>> expired async-jobs
>> 2017-01-18 12:18:15,993 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-7ccabf83) (logid:2b0f34c4) End cleanup
>> expired async-jobs
>> 2017-01-18 12:18:16,046 DEBUG
>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> (RouterStatusMonitor-1:ctx-fc5408a0) (logid:bbd625c1) Found 0 routers
>> to update status.
>> 2017-01-18 12:18:16,048 DEBUG
>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> (RouterStatusMonitor-1:ctx-fc5408a0) (logid:bbd625c1) Found 0 VPC
>> networks to update Redundant State.
>> 2017-01-18 12:18:16,049 DEBUG
>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> (RouterStatusMonitor-1:ctx-fc5408a0) (logid:bbd625c1) Found 0 networks
>> to update RvR status.
>> 2017-01-18 12:18:16,097 DEBUG
>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> (RouterStatusMonitor-1:ctx-2c697f6b) (logid:cfa955f2) Found 0 routers
>> to update status.
>> 2017-01-18 12:18:16,099 DEBUG
>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> (RouterStatusMonitor-1:ctx-2c697f6b) (logid:cfa955f2) Found 0 VPC
>> networks to update Redundant State.
>> 2017-01-18 12:18:16,101 DEBUG
>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> (RouterStatusMonitor-1:ctx-2c697f6b) (logid:cfa955f2) Found 0 networks
>> to update RvR status.
>> 2017-01-18 12:18:24,389 DEBUG [c.c.h.d.HostDaoImpl]
>> (ClusteredAgentManager
>> Timer:ctx-d699e7ed) (logid:e10673e8) Resetting hosts suitable for
>> reconnect
>> 2017-01-18 12:18:24,392 DEBUG [c.c.h.d.HostDaoImpl]
>> (ClusteredAgentManager
>> Timer:ctx-d699e7ed) (logid:e10673e8) Completed resetting hosts
>> suitable for reconnect
>> 2017-01-18 12:18:24,392 DEBUG [c.c.h.d.HostDaoImpl]
>> (ClusteredAgentManager
>> Timer:ctx-d699e7ed) (logid:e10673e8) Acquiring hosts for clusters
>> already owned by this management server
>> 2017-01-18 12:18:24,392 DEBUG [c.c.h.d.HostDaoImpl]
>> (ClusteredAgentManager
>> Timer:ctx-d699e7ed) (logid:e10673e8) Completed acquiring hosts for
>> clusters already owned by this management server
>> 2017-01-18 12:18:24,392 DEBUG [c.c.h.d.HostDaoImpl]
>> (ClusteredAgentManager
>> Timer:ctx-d699e7ed) (logid:e10673e8) Acquiring hosts for clusters not
>> owned by any management server
>> 2017-01-18 12:18:24,393 DEBUG [c.c.h.d.HostDaoImpl]
>> (ClusteredAgentManager
>> Timer:ctx-d699e7ed) (logid:e10673e8) Completed acquiring hosts for
>> clusters not owned by any management server
>> 2017-01-18 12:18:24,807 DEBUG [c.c.a.m.AgentManagerImpl]
>> (AgentManager-Handler-6:null) (logid:) SeqA 3-438485: Processing Seq
>> 3-438485: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
>> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,
>> "_loadInfo":"{\n
>> \"connections\": []\n}","wait":0}}] }
>> 2017-01-18 12:18:24,870 DEBUG [c.c.a.m.AgentManagerImpl]
>> (AgentManager-Handler-6:null) (logid:) SeqA 3-438485: Sending Seq
>> 3-438485: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1, Flags:
>> 100010, [{"com.cloud.agent.api.AgentControlAnswer":{"result":tru
>>
>>> Can you please share your management_server logs
>>>
>>> On Tue, Jan 17, 2017 at 11:47 PM, 조대Ø•
>>> <ca...@renet.kr> wrote:
>>>
>>>> Hi, all
>>>>
>>>> I am facing secondary storage problem.
>>>>
>>>> I am using NFS Server to use primary and secondary storage.
>>>>
>>>> After rebooting the host,
>>>>
>>>> In Dashboard Cloudstack , it says Secondary Storage 0.00 KB / 0.00
>>>> KB
>>>>
>>>> Primary Storage is okay.
>>>>
>>>>
>>>>
>>>> After all, system VMs are not up. It was working fine before
>>>> rebooting.
>>>>
>>>>
>>>>
>>>> I have tried several times of restart of Management service.
>>>>
>>>>
>>>>
>>>> Any ideas?
>>>>
>>>>
>>>>
>>>> Thanks,
>>>>
>>>>
>>>
>>
>>
>>
>
>
>
Dag.Sonstebo@shapeblue.com
www.shapeblue.com
53 Chandos Place, Covent Garden, London WC2N 4HSUK
@shapeblue
RE: Cloudstack 4.7 Secondary Storage problem
Posted by me...@students.iitmandi.ac.in.
Also when you say mount from ssvm. Does it mean this?
Log into ssvm and run "mount":- the reply is down below. I am not able to
understand much even from the documentation,
I am sorry for the lack of full knowledge in this concern.
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
udev on /dev type devtmpfs (rw,relatime,size=10240k,nr_inodes=62128,mode=755)
devpts on /dev/pts type devpts
(rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,noexec,relatime,size=50896k,mode=755)
/dev/disk/by-uuid/30c81d3d-ee9f-4a88-81c1-5f349b22ba1d on / type ext4
(rw,relatime,errors=remount-ro,user_xattr,barrier=1,data=ordered)
tmpfs on /run/lock type tmpfs (rw,nosuid,nodev,noexec,relatime,size=5120k)
tmpfs on /run/shm type tmpfs (rw,nosuid,nodev,noexec,relatime,size=211960k)
/dev/vda1 on /boot type ext4 (rw,relatime,user_xattr,barrier=1,data=ordered)
/dev/vda6 on /home type ext4 (rw,relatime,user_xattr,barrier=1,data=ordered)
/dev/vda8 on /opt type ext4 (rw,relatime,user_xattr,barrier=1,data=ordered)
/dev/vda11 on /tmp type ext4 (rw,relatime,user_xattr,barrier=1,data=ordered)
/dev/vda7 on /usr type ext4 (rw,relatime,user_xattr,barrier=1,data=ordered)
/dev/vda9 on /var type ext4 (rw,relatime,user_xattr,barrier=1,data=ordered)
/dev/vda10 on /var/log type ext4
(rw,relatime,user_xattr,barrier=1,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
> Hi Merlin,
>
> To troubleshoot, try mounting the NFS yourself from the SSVM and get that
> working.
>
> the likely problem is permissions on the NFS mounts. Either the IP of the
> SSVM is not allowed or the permissions don't include no_root_squash (or
> equivalent) the SSVM is probably connecting as 'cloud', so connections
> have to be mapped to 'root'. Finally the SSVM only reliably supports
> NFSv3
>
>
>
>
> Kind regards,
>
> Paul Angus
>
> paul.angus@shapeblue.com
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London WC2N 4HSUK
> @shapeblue
>
>
>
>
> -----Original Message-----
> From: merlin@students.iitmandi.ac.in
> [mailto:merlin@students.iitmandi.ac.in]
> Sent: 18 January 2017 08:48
> To: users@cloudstack.apache.org
> Subject: Re: Cloudstack 4.7 Secondary Storage problem
>
> And this is the result of check.sh
>
> 2 packets transmitted, 2 packets received, 0% packet loss round-trip
> min/avg/max/stddev = 0.400/0.550/0.701/0.151 ms
> Good: Can ping DNS server
> ================================================
> Good: DNS resolves download.cloud.com
> ================================================
> ERROR: NFS is not currently mounted
> Try manually mounting from inside the VM NFS server is 10.8.0.1 PING
> 10.8.0.1 (10.8.0.1): 48 data bytes
> 56 bytes from 10.8.0.1: icmp_seq=0 ttl=64 time=0.456 ms
> 56 bytes from 10.8.0.1: icmp_seq=1 ttl=64 time=0.708 ms
> --- 10.8.0.1 ping statistics ---
> 2 packets transmitted, 2 packets received, 0% packet loss round-trip
> min/avg/max/stddev = 0.456/0.582/0.708/0.126 ms
> Good: Can ping nfs server
> ================================================
> Management server is 10.8.19.111. Checking connectivity.
> Good: Can connect to management server port 8250
> ================================================
> Good: Java process is running
> ================================================
> Tests Complete. Look for ERROR or WARNING above.
>
>
> The problem is that while installation I did mount nfs?
> I dont understand what else to do next
>
>
>
>> Hi, even I am facing the same problem.. and here are the logs from
>> today:
>>
>>
>>
>> 2017-01-18 12:16:46,100 DEBUG
>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> (RouterStatusMonitor-1:ctx-bc5cb4cd) (logid:0ffac658) Found 0 networks
>> to update RvR status.
>> 2017-01-18 12:16:46,200 DEBUG [c.c.a.m.AgentManagerImpl]
>> (AgentManager-Handler-11:null) (logid:) Ping from 2(s-2-VM)
>> 2017-01-18 12:16:47,521 DEBUG [c.c.a.m.AgentManagerImpl]
>> (AgentManager-Handler-15:null) (logid:) Ping from 3(v-1-VM)
>> 2017-01-18 12:16:49,834 DEBUG [c.c.a.m.AgentManagerImpl]
>> (AgentManager-Handler-9:null) (logid:) SeqA 3-438472: Processing Seq
>> 3-438472: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
>> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,
>> "_loadInfo":"{\n
>> \"connections\": []\n}","wait":0}}] }
>> 2017-01-18 12:16:49,887 DEBUG [c.c.a.m.AgentManagerImpl]
>> (AgentManager-Handler-9:null) (logid:) SeqA 3-438472: Sending Seq
>> 3-438472: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1, Flags:
>> 100010,
>> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}]
>> }
>> 2017-01-18 12:16:54,390 DEBUG [c.c.h.d.HostDaoImpl]
>> (ClusteredAgentManager
>> Timer:ctx-8464e4b3) (logid:1f15c721) Resetting hosts suitable for
>> reconnect
>> 2017-01-18 12:16:54,392 DEBUG [c.c.h.d.HostDaoImpl]
>> (ClusteredAgentManager
>> Timer:ctx-8464e4b3) (logid:1f15c721) Completed resetting hosts
>> suitable for reconnect
>> 2017-01-18 12:16:54,392 DEBUG [c.c.h.d.HostDaoImpl]
>> (ClusteredAgentManager
>> Timer:ctx-8464e4b3) (logid:1f15c721) Acquiring hosts for clusters
>> already owned by this management server
>> 2017-01-18 12:16:54,393 DEBUG [c.c.h.d.HostDaoImpl]
>> (ClusteredAgentManager
>> Timer:ctx-8464e4b3) (logid:1f15c721) Completed acquiring hosts for
>> clusters already owned by this management server
>> 2017-01-18 12:16:54,393 DEBUG [c.c.h.d.HostDaoImpl]
>> (ClusteredAgentManager
>> Timer:ctx-8464e4b3) (logid:1f15c721) Acquiring hosts for clusters not
>> owned by any management server
>> 2017-01-18 12:16:54,394 DEBUG [c.c.h.d.HostDaoImpl]
>> (ClusteredAgentManager
>> Timer:ctx-8464e4b3) (logid:1f15c721) Completed acquiring hosts for
>> clusters not owned by any management server
>> 2017-01-18 12:16:54,795 DEBUG [c.c.a.m.AgentManagerImpl]
>> (AgentManager-Handler-5:null) (logid:) SeqA 3-438473: Processing Seq
>> 3-438473: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
>> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,
>> "_loadInfo":"{\n
>> \"connections\": []\n}","wait":0}}] }
>> 2017-01-18 12:16:54,854 DEBUG [c.c.a.m.AgentManagerImpl]
>> (AgentManager-Handler-5:null) (logid:) SeqA 3-438473: Sending Seq
>> 3-438473: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1, Flags:
>> 100010,
>> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}]
>> }
>> 2017-01-18 12:16:55,984 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-d0469453) (logid:af5d4f47) Begin cleanup
>> expired async-jobs
>> 2017-01-18 12:16:55,991 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-d0469453) (logid:af5d4f47) End cleanup
>> expired async-jobs
>> 2017-01-18 12:16:59,245 DEBUG [c.c.s.StatsCollector]
>> (StatsCollector-1:ctx-68069dff) (logid:a7a6974d) VmStatsCollector is
>> running...
>> 2017-01-18 12:17:00,841 DEBUG [o.a.c.s.SecondaryStorageManagerImpl]
>> (secstorage-1:ctx-4e101aa5) (logid:65f1f0cb) Zone 1 is ready to launch
>> secondary storage VM
>> 2017-01-18 12:17:01,095 DEBUG [c.c.c.ConsoleProxyManagerImpl]
>> (consoleproxy-1:ctx-393828fa) (logid:7b3d0491) Zone 1 is ready to
>> launch console proxy
>> 2017-01-18 12:17:03,283 DEBUG [c.c.s.StatsCollector]
>> (StatsCollector-3:ctx-cc1f58a1) (logid:3ae41340) HostStatsCollector is
>> running...
>> 2017-01-18 12:17:03,343 DEBUG [c.c.a.t.Request]
>> (StatsCollector-3:ctx-cc1f58a1) (logid:3ae41340) Seq
>> 1-6227070909769882804: Received: { Ans: , MgmtId: 279278805450954,
>> via:
>> 1(iuatc), Ver: v1, Flags: 10, { GetHostStatsAnswer } }
>> 2017-01-18 12:17:04,796 DEBUG [c.c.a.m.AgentManagerImpl]
>> (AgentManager-Handler-10:null) (logid:) SeqA 3-438474: Processing Seq
>> 3-438474: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
>> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,
>> "_loadInfo":"{\n
>> \"connections\": []\n}","wait":0}}] }
>> 2017-01-18 12:17:04,847 DEBUG [c.c.a.m.AgentManagerImpl]
>> (AgentManager-Handler-10:null) (logid:) SeqA 3-438474: Sending Seq
>> 3-438474: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1, Flags:
>> 100010,
>> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}]
>> }
>> 2017-01-18 12:17:05,985 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-7f0de3b8) (logid:e9bbd63a) Begin cleanup
>> expired async-jobs
>> 2017-01-18 12:17:05,991 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-7f0de3b8) (logid:e9bbd63a) End cleanup
>> expired async-jobs
>> 2017-01-18 12:17:14,798 DEBUG [c.c.a.m.AgentManagerImpl]
>> (AgentManager-Handler-8:null) (logid:) SeqA 3-438475: Processing Seq
>> 3-438475: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
>> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,
>> "_loadInfo":"{\n
>> \"connections\": []\n}","wait":0}}] }
>> 2017-01-18 12:17:14,857 DEBUG [c.c.a.m.AgentManagerImpl]
>> (AgentManager-Handler-8:null) (logid:) SeqA 3-438475: Sending Seq
>> 3-438475: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1, Flags:
>> 100010,
>> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}]
>> }
>> 2017-01-18 12:17:15,984 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-5682fe3c) (logid:901efb52) Begin cleanup
>> expired async-jobs
>> 2017-01-18 12:17:15,990 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-5682fe3c) (logid:901efb52) End cleanup
>> expired async-jobs
>> 2017-01-18 12:17:16,046 DEBUG
>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> (RouterStatusMonitor-1:ctx-7ad0b18a) (logid:b5b19f2b) Found 0 routers
>> to update status.
>> 2017-01-18 12:17:16,047 DEBUG
>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> (RouterStatusMonitor-1:ctx-7ad0b18a) (logid:b5b19f2b) Found 0 VPC
>> networks to update Redundant State.
>> 2017-01-18 12:17:16,049 DEBUG
>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> (RouterStatusMonitor-1:ctx-7ad0b18a) (logid:b5b19f2b) Found 0 networks
>> to update RvR status.
>> 2017-01-18 12:17:16,097 DEBUG
>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> (RouterStatusMonitor-1:ctx-10a8aefd) (logid:b09ffa05) Found 0 routers
>> to update status.
>> 2017-01-18 12:17:16,099 DEBUG
>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> (RouterStatusMonitor-1:ctx-10a8aefd) (logid:b09ffa05) Found 0 VPC
>> networks to update Redundant State.
>> 2017-01-18 12:17:16,101 DEBUG
>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> (RouterStatusMonitor-1:ctx-10a8aefd) (logid:b09ffa05) Found 0 networks
>> to update RvR status.
>> 2017-01-18 12:17:19,799 DEBUG [c.c.a.m.AgentManagerImpl]
>> (AgentManager-Handler-3:null) (logid:) SeqA 3-438476: Processing Seq
>> 3-438476: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
>> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,
>> "_loadInfo":"{\n
>> \"connections\": []\n}","wait":0}}] }
>> 2017-01-18 12:17:19,925 DEBUG [c.c.a.m.AgentManagerImpl]
>> (AgentManager-Handler-3:null) (logid:) SeqA 3-438476: Sending Seq
>> 3-438476: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1, Flags:
>> 100010,
>> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}]
>> }
>> 2017-01-18 12:17:25,985 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-a4c8ceb4) (logid:d0d85ed6) Begin cleanup
>> expired async-jobs
>> 2017-01-18 12:17:25,991 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-a4c8ceb4) (logid:d0d85ed6) End cleanup
>> expired async-jobs
>> 2017-01-18 12:17:29,800 DEBUG [c.c.a.m.AgentManagerImpl]
>> (AgentManager-Handler-7:null) (logid:) SeqA 3-438477: Processing Seq
>> 3-438477: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
>> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,
>> "_loadInfo":"{\n
>> \"connections\": []\n}","wait":0}}] }
>> 2017-01-18 12:17:29,860 DEBUG [c.c.a.m.AgentManagerImpl]
>> (AgentManager-Handler-7:null) (logid:) SeqA 3-438477: Sending Seq
>> 3-438477: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1, Flags:
>> 100010,
>> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}]
>> }
>> 2017-01-18 12:17:30,844 DEBUG [o.a.c.s.SecondaryStorageManagerImpl]
>> (secstorage-1:ctx-f2d92c7a) (logid:6bcbea1f) Zone 1 is ready to launch
>> secondary storage VM
>> 2017-01-18 12:17:31,095 DEBUG [c.c.c.ConsoleProxyManagerImpl]
>> (consoleproxy-1:ctx-5c5c1698) (logid:a199e87c) Zone 1 is ready to
>> launch console proxy
>> 2017-01-18 12:17:34,801 DEBUG [c.c.a.m.AgentManagerImpl]
>> (AgentManager-Handler-4:null) (logid:) SeqA 3-438478: Processing Seq
>> 3-438478: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
>> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,
>> "_loadInfo":"{\n
>> \"connections\": []\n}","wait":0}}] }
>> 2017-01-18 12:17:34,853 DEBUG [c.c.a.m.AgentManagerImpl]
>> (AgentManager-Handler-4:null) (logid:) SeqA 3-438478: Sending Seq
>> 3-438478: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1, Flags:
>> 100010,
>> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}]
>> }
>> 2017-01-18 12:17:35,534 DEBUG [c.c.s.StatsCollector]
>> (StatsCollector-4:ctx-37da523c) (logid:aecb957a) AutoScaling Monitor
>> is running...
>> 2017-01-18 12:17:35,984 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-d031d5da) (logid:722c8ae3) Begin cleanup
>> expired async-jobs
>> 2017-01-18 12:17:35,990 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-d031d5da) (logid:722c8ae3) End cleanup
>> expired async-jobs
>> 2017-01-18 12:17:44,802 DEBUG [c.c.a.m.AgentManagerImpl]
>> (AgentManager-Handler-1:null) (logid:) SeqA 3-438479: Processing Seq
>> 3-438479: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
>> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,
>> "_loadInfo":"{\n
>> \"connections\": []\n}","wait":0}}] }
>> 2017-01-18 12:17:44,863 DEBUG [c.c.a.m.AgentManagerImpl]
>> (AgentManager-Handler-1:null) (logid:) SeqA 3-438479: Sending Seq
>> 3-438479: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1, Flags:
>> 100010,
>> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}]
>> }
>> 2017-01-18 12:17:45,754 DEBUG [c.c.a.m.AgentManagerImpl]
>> (AgentManager-Handler-12:null) (logid:) Ping from 1(iuatc)
>> 2017-01-18 12:17:45,754 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
>> (AgentManager-Handler-12:null) (logid:) Process host VM state report
>> from ping process. host: 1
>> 2017-01-18 12:17:45,760 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
>> (AgentManager-Handler-12:null) (logid:) Process VM state report. host:
>> 1, number of records in report: 2
>> 2017-01-18 12:17:45,760 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
>> (AgentManager-Handler-12:null) (logid:) VM state report. host: 1, vm
>> id:
>> 1, power state: PowerOn
>> 2017-01-18 12:17:45,763 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
>> (AgentManager-Handler-12:null) (logid:) VM power state does not
>> change, skip DB writing. vm id: 1
>> 2017-01-18 12:17:45,763 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
>> (AgentManager-Handler-12:null) (logid:) VM state report. host: 1, vm
>> id:
>> 2, power state: PowerOn
>> 2017-01-18 12:17:45,766 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
>> (AgentManager-Handler-12:null) (logid:) VM power state does not
>> change, skip DB writing. vm id: 2
>> 2017-01-18 12:17:45,769 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
>> (AgentManager-Handler-12:null) (logid:) Done with process of VM state
>> report. host: 1
>> 2017-01-18 12:17:45,985 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-b39e7e84) (logid:d0f90660) Begin cleanup
>> expired async-jobs
>> 2017-01-18 12:17:45,991 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-b39e7e84) (logid:d0f90660) End cleanup
>> expired async-jobs
>> 2017-01-18 12:17:46,046 DEBUG
>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> (RouterStatusMonitor-1:ctx-36fbb5c2) (logid:d346d004) Found 0 routers
>> to update status.
>> 2017-01-18 12:17:46,047 DEBUG
>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> (RouterStatusMonitor-1:ctx-36fbb5c2) (logid:d346d004) Found 0 VPC
>> networks to update Redundant State.
>> 2017-01-18 12:17:46,049 DEBUG
>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> (RouterStatusMonitor-1:ctx-36fbb5c2) (logid:d346d004) Found 0 networks
>> to update RvR status.
>> 2017-01-18 12:17:46,097 DEBUG
>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> (RouterStatusMonitor-1:ctx-16234697) (logid:beb5fec7) Found 0 routers
>> to update status.
>> 2017-01-18 12:17:46,099 DEBUG
>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> (RouterStatusMonitor-1:ctx-16234697) (logid:beb5fec7) Found 0 VPC
>> networks to update Redundant State.
>> 2017-01-18 12:17:46,100 DEBUG
>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> (RouterStatusMonitor-1:ctx-16234697) (logid:beb5fec7) Found 0 networks
>> to update RvR status.
>> 2017-01-18 12:17:46,201 DEBUG [c.c.a.m.AgentManagerImpl]
>> (AgentManager-Handler-2:null) (logid:) Ping from 2(s-2-VM)
>> 2017-01-18 12:17:47,521 DEBUG [c.c.a.m.AgentManagerImpl]
>> (AgentManager-Handler-14:null) (logid:) Ping from 3(v-1-VM)
>> 2017-01-18 12:17:49,842 DEBUG [c.c.a.m.AgentManagerImpl]
>> (AgentManager-Handler-13:null) (logid:) SeqA 3-438481: Processing Seq
>> 3-438481: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
>> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,
>> "_loadInfo":"{\n
>> \"connections\": []\n}","wait":0}}] }
>> 2017-01-18 12:17:49,897 DEBUG [c.c.a.m.AgentManagerImpl]
>> (AgentManager-Handler-13:null) (logid:) SeqA 3-438481: Sending Seq
>> 3-438481: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1, Flags:
>> 100010,
>> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}]
>> }
>> 2017-01-18 12:17:55,985 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-c0db35b5) (logid:c304f172) Begin cleanup
>> expired async-jobs
>> 2017-01-18 12:17:55,991 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-c0db35b5) (logid:c304f172) End cleanup
>> expired async-jobs
>> 2017-01-18 12:17:59,250 DEBUG [c.c.s.StatsCollector]
>> (StatsCollector-6:ctx-eec84206) (logid:b88e0ce8) VmStatsCollector is
>> running...
>> 2017-01-18 12:17:59,803 DEBUG [c.c.a.m.AgentManagerImpl]
>> (AgentManager-Handler-11:null) (logid:) SeqA 3-438482: Processing Seq
>> 3-438482: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
>> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,
>> "_loadInfo":"{\n
>> \"connections\": []\n}","wait":0}}] }
>> 2017-01-18 12:17:59,857 DEBUG [c.c.a.m.AgentManagerImpl]
>> (AgentManager-Handler-11:null) (logid:) SeqA 3-438482: Sending Seq
>> 3-438482: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1, Flags:
>> 100010,
>> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}]
>> }
>> 2017-01-18 12:18:00,843 DEBUG [o.a.c.s.SecondaryStorageManagerImpl]
>> (secstorage-1:ctx-f45be089) (logid:934bd9f7) Zone 1 is ready to launch
>> secondary storage VM
>> 2017-01-18 12:18:01,094 DEBUG [c.c.c.ConsoleProxyManagerImpl]
>> (consoleproxy-1:ctx-4b11dfe0) (logid:cce52dbb) Zone 1 is ready to
>> launch console proxy
>> 2017-01-18 12:18:03,345 DEBUG [c.c.s.StatsCollector]
>> (StatsCollector-2:ctx-c743d4ee) (logid:c1f62e31) HostStatsCollector is
>> running...
>> 2017-01-18 12:18:03,406 DEBUG [c.c.a.t.Request]
>> (StatsCollector-2:ctx-c743d4ee) (logid:c1f62e31) Seq
>> 1-6227070909769882805: Received: { Ans: , MgmtId: 279278805450954,
>> via:
>> 1(iuatc), Ver: v1, Flags: 10, { GetHostStatsAnswer } }
>> 2017-01-18 12:18:04,804 DEBUG [c.c.a.m.AgentManagerImpl]
>> (AgentManager-Handler-9:null) (logid:) SeqA 3-438483: Processing Seq
>> 3-438483: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
>> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,
>> "_loadInfo":"{\n
>> \"connections\": []\n}","wait":0}}] }
>> 2017-01-18 12:18:04,858 DEBUG [c.c.a.m.AgentManagerImpl]
>> (AgentManager-Handler-9:null) (logid:) SeqA 3-438483: Sending Seq
>> 3-438483: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1, Flags:
>> 100010,
>> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}]
>> }
>> 2017-01-18 12:18:05,985 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-149380f2) (logid:53d166f9) Begin cleanup
>> expired async-jobs
>> 2017-01-18 12:18:05,990 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-149380f2) (logid:53d166f9) End cleanup
>> expired async-jobs
>> 2017-01-18 12:18:14,805 DEBUG [c.c.a.m.AgentManagerImpl]
>> (AgentManager-Handler-5:null) (logid:) SeqA 3-438484: Processing Seq
>> 3-438484: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
>> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,
>> "_loadInfo":"{\n
>> \"connections\": []\n}","wait":0}}] }
>> 2017-01-18 12:18:14,951 DEBUG [c.c.a.m.AgentManagerImpl]
>> (AgentManager-Handler-5:null) (logid:) SeqA 3-438484: Sending Seq
>> 3-438484: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1, Flags:
>> 100010,
>> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}]
>> }
>> 2017-01-18 12:18:15,985 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-7ccabf83) (logid:2b0f34c4) Begin cleanup
>> expired async-jobs
>> 2017-01-18 12:18:15,993 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-7ccabf83) (logid:2b0f34c4) End cleanup
>> expired async-jobs
>> 2017-01-18 12:18:16,046 DEBUG
>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> (RouterStatusMonitor-1:ctx-fc5408a0) (logid:bbd625c1) Found 0 routers
>> to update status.
>> 2017-01-18 12:18:16,048 DEBUG
>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> (RouterStatusMonitor-1:ctx-fc5408a0) (logid:bbd625c1) Found 0 VPC
>> networks to update Redundant State.
>> 2017-01-18 12:18:16,049 DEBUG
>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> (RouterStatusMonitor-1:ctx-fc5408a0) (logid:bbd625c1) Found 0 networks
>> to update RvR status.
>> 2017-01-18 12:18:16,097 DEBUG
>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> (RouterStatusMonitor-1:ctx-2c697f6b) (logid:cfa955f2) Found 0 routers
>> to update status.
>> 2017-01-18 12:18:16,099 DEBUG
>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> (RouterStatusMonitor-1:ctx-2c697f6b) (logid:cfa955f2) Found 0 VPC
>> networks to update Redundant State.
>> 2017-01-18 12:18:16,101 DEBUG
>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> (RouterStatusMonitor-1:ctx-2c697f6b) (logid:cfa955f2) Found 0 networks
>> to update RvR status.
>> 2017-01-18 12:18:24,389 DEBUG [c.c.h.d.HostDaoImpl]
>> (ClusteredAgentManager
>> Timer:ctx-d699e7ed) (logid:e10673e8) Resetting hosts suitable for
>> reconnect
>> 2017-01-18 12:18:24,392 DEBUG [c.c.h.d.HostDaoImpl]
>> (ClusteredAgentManager
>> Timer:ctx-d699e7ed) (logid:e10673e8) Completed resetting hosts
>> suitable for reconnect
>> 2017-01-18 12:18:24,392 DEBUG [c.c.h.d.HostDaoImpl]
>> (ClusteredAgentManager
>> Timer:ctx-d699e7ed) (logid:e10673e8) Acquiring hosts for clusters
>> already owned by this management server
>> 2017-01-18 12:18:24,392 DEBUG [c.c.h.d.HostDaoImpl]
>> (ClusteredAgentManager
>> Timer:ctx-d699e7ed) (logid:e10673e8) Completed acquiring hosts for
>> clusters already owned by this management server
>> 2017-01-18 12:18:24,392 DEBUG [c.c.h.d.HostDaoImpl]
>> (ClusteredAgentManager
>> Timer:ctx-d699e7ed) (logid:e10673e8) Acquiring hosts for clusters not
>> owned by any management server
>> 2017-01-18 12:18:24,393 DEBUG [c.c.h.d.HostDaoImpl]
>> (ClusteredAgentManager
>> Timer:ctx-d699e7ed) (logid:e10673e8) Completed acquiring hosts for
>> clusters not owned by any management server
>> 2017-01-18 12:18:24,807 DEBUG [c.c.a.m.AgentManagerImpl]
>> (AgentManager-Handler-6:null) (logid:) SeqA 3-438485: Processing Seq
>> 3-438485: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
>> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,
>> "_loadInfo":"{\n
>> \"connections\": []\n}","wait":0}}] }
>> 2017-01-18 12:18:24,870 DEBUG [c.c.a.m.AgentManagerImpl]
>> (AgentManager-Handler-6:null) (logid:) SeqA 3-438485: Sending Seq
>> 3-438485: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1, Flags:
>> 100010, [{"com.cloud.agent.api.AgentControlAnswer":{"result":tru
>>
>>> Can you please share your management_server logs
>>>
>>> On Tue, Jan 17, 2017 at 11:47 PM, ì¡°ë\u0152\u20ací\u02dc\u2022
>>> <ca...@renet.kr> wrote:
>>>
>>>> Hi, all
>>>>
>>>> I am facing secondary storage problem.
>>>>
>>>> I am using NFS Server to use primary and secondary storage.
>>>>
>>>> After rebooting the host,
>>>>
>>>> In Dashboard Cloudstack , it says Secondary Storage 0.00 KB / 0.00
>>>> KB
>>>>
>>>> Primary Storage is okay.
>>>>
>>>>
>>>>
>>>> After all, system VMs are not up. It was working fine before
>>>> rebooting.
>>>>
>>>>
>>>>
>>>> I have tried several times of restart of Management service.
>>>>
>>>>
>>>>
>>>> Any ideas?
>>>>
>>>>
>>>>
>>>> Thanks,
>>>>
>>>>
>>>
>>
>>
>>
>
>
>
RE: Cloudstack 4.7 Secondary Storage problem
Posted by me...@students.iitmandi.ac.in.
Hi paul,
Thanx for your suggestion.
Is it also possible that the if the global setting for
secstorage.allowed.internal.sites is not set to proper ip?? Well I have
set it to the hosts ip(which ismy management server and hypervisor both)?
> Hi Merlin,
>
> To troubleshoot, try mounting the NFS yourself from the SSVM and get that
> working.
>
> the likely problem is permissions on the NFS mounts. Either the IP of the
> SSVM is not allowed or the permissions don't include no_root_squash (or
> equivalent) the SSVM is probably connecting as 'cloud', so connections
> have to be mapped to 'root'. Finally the SSVM only reliably supports
> NFSv3
>
>
>
>
> Kind regards,
>
> Paul Angus
>
> paul.angus@shapeblue.com
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London WC2N 4HSUK
> @shapeblue
>
>
>
>
> -----Original Message-----
> From: merlin@students.iitmandi.ac.in
> [mailto:merlin@students.iitmandi.ac.in]
> Sent: 18 January 2017 08:48
> To: users@cloudstack.apache.org
> Subject: Re: Cloudstack 4.7 Secondary Storage problem
>
> And this is the result of check.sh
>
> 2 packets transmitted, 2 packets received, 0% packet loss round-trip
> min/avg/max/stddev = 0.400/0.550/0.701/0.151 ms
> Good: Can ping DNS server
> ================================================
> Good: DNS resolves download.cloud.com
> ================================================
> ERROR: NFS is not currently mounted
> Try manually mounting from inside the VM NFS server is 10.8.0.1 PING
> 10.8.0.1 (10.8.0.1): 48 data bytes
> 56 bytes from 10.8.0.1: icmp_seq=0 ttl=64 time=0.456 ms
> 56 bytes from 10.8.0.1: icmp_seq=1 ttl=64 time=0.708 ms
> --- 10.8.0.1 ping statistics ---
> 2 packets transmitted, 2 packets received, 0% packet loss round-trip
> min/avg/max/stddev = 0.456/0.582/0.708/0.126 ms
> Good: Can ping nfs server
> ================================================
> Management server is 10.8.19.111. Checking connectivity.
> Good: Can connect to management server port 8250
> ================================================
> Good: Java process is running
> ================================================
> Tests Complete. Look for ERROR or WARNING above.
>
>
> The problem is that while installation I did mount nfs?
> I dont understand what else to do next
>
>
>
>> Hi, even I am facing the same problem.. and here are the logs from
>> today:
>>
>>
>>
>> 2017-01-18 12:16:46,100 DEBUG
>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> (RouterStatusMonitor-1:ctx-bc5cb4cd) (logid:0ffac658) Found 0 networks
>> to update RvR status.
>> 2017-01-18 12:16:46,200 DEBUG [c.c.a.m.AgentManagerImpl]
>> (AgentManager-Handler-11:null) (logid:) Ping from 2(s-2-VM)
>> 2017-01-18 12:16:47,521 DEBUG [c.c.a.m.AgentManagerImpl]
>> (AgentManager-Handler-15:null) (logid:) Ping from 3(v-1-VM)
>> 2017-01-18 12:16:49,834 DEBUG [c.c.a.m.AgentManagerImpl]
>> (AgentManager-Handler-9:null) (logid:) SeqA 3-438472: Processing Seq
>> 3-438472: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
>> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,
>> "_loadInfo":"{\n
>> \"connections\": []\n}","wait":0}}] }
>> 2017-01-18 12:16:49,887 DEBUG [c.c.a.m.AgentManagerImpl]
>> (AgentManager-Handler-9:null) (logid:) SeqA 3-438472: Sending Seq
>> 3-438472: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1, Flags:
>> 100010,
>> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}]
>> }
>> 2017-01-18 12:16:54,390 DEBUG [c.c.h.d.HostDaoImpl]
>> (ClusteredAgentManager
>> Timer:ctx-8464e4b3) (logid:1f15c721) Resetting hosts suitable for
>> reconnect
>> 2017-01-18 12:16:54,392 DEBUG [c.c.h.d.HostDaoImpl]
>> (ClusteredAgentManager
>> Timer:ctx-8464e4b3) (logid:1f15c721) Completed resetting hosts
>> suitable for reconnect
>> 2017-01-18 12:16:54,392 DEBUG [c.c.h.d.HostDaoImpl]
>> (ClusteredAgentManager
>> Timer:ctx-8464e4b3) (logid:1f15c721) Acquiring hosts for clusters
>> already owned by this management server
>> 2017-01-18 12:16:54,393 DEBUG [c.c.h.d.HostDaoImpl]
>> (ClusteredAgentManager
>> Timer:ctx-8464e4b3) (logid:1f15c721) Completed acquiring hosts for
>> clusters already owned by this management server
>> 2017-01-18 12:16:54,393 DEBUG [c.c.h.d.HostDaoImpl]
>> (ClusteredAgentManager
>> Timer:ctx-8464e4b3) (logid:1f15c721) Acquiring hosts for clusters not
>> owned by any management server
>> 2017-01-18 12:16:54,394 DEBUG [c.c.h.d.HostDaoImpl]
>> (ClusteredAgentManager
>> Timer:ctx-8464e4b3) (logid:1f15c721) Completed acquiring hosts for
>> clusters not owned by any management server
>> 2017-01-18 12:16:54,795 DEBUG [c.c.a.m.AgentManagerImpl]
>> (AgentManager-Handler-5:null) (logid:) SeqA 3-438473: Processing Seq
>> 3-438473: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
>> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,
>> "_loadInfo":"{\n
>> \"connections\": []\n}","wait":0}}] }
>> 2017-01-18 12:16:54,854 DEBUG [c.c.a.m.AgentManagerImpl]
>> (AgentManager-Handler-5:null) (logid:) SeqA 3-438473: Sending Seq
>> 3-438473: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1, Flags:
>> 100010,
>> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}]
>> }
>> 2017-01-18 12:16:55,984 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-d0469453) (logid:af5d4f47) Begin cleanup
>> expired async-jobs
>> 2017-01-18 12:16:55,991 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-d0469453) (logid:af5d4f47) End cleanup
>> expired async-jobs
>> 2017-01-18 12:16:59,245 DEBUG [c.c.s.StatsCollector]
>> (StatsCollector-1:ctx-68069dff) (logid:a7a6974d) VmStatsCollector is
>> running...
>> 2017-01-18 12:17:00,841 DEBUG [o.a.c.s.SecondaryStorageManagerImpl]
>> (secstorage-1:ctx-4e101aa5) (logid:65f1f0cb) Zone 1 is ready to launch
>> secondary storage VM
>> 2017-01-18 12:17:01,095 DEBUG [c.c.c.ConsoleProxyManagerImpl]
>> (consoleproxy-1:ctx-393828fa) (logid:7b3d0491) Zone 1 is ready to
>> launch console proxy
>> 2017-01-18 12:17:03,283 DEBUG [c.c.s.StatsCollector]
>> (StatsCollector-3:ctx-cc1f58a1) (logid:3ae41340) HostStatsCollector is
>> running...
>> 2017-01-18 12:17:03,343 DEBUG [c.c.a.t.Request]
>> (StatsCollector-3:ctx-cc1f58a1) (logid:3ae41340) Seq
>> 1-6227070909769882804: Received: { Ans: , MgmtId: 279278805450954,
>> via:
>> 1(iuatc), Ver: v1, Flags: 10, { GetHostStatsAnswer } }
>> 2017-01-18 12:17:04,796 DEBUG [c.c.a.m.AgentManagerImpl]
>> (AgentManager-Handler-10:null) (logid:) SeqA 3-438474: Processing Seq
>> 3-438474: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
>> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,
>> "_loadInfo":"{\n
>> \"connections\": []\n}","wait":0}}] }
>> 2017-01-18 12:17:04,847 DEBUG [c.c.a.m.AgentManagerImpl]
>> (AgentManager-Handler-10:null) (logid:) SeqA 3-438474: Sending Seq
>> 3-438474: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1, Flags:
>> 100010,
>> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}]
>> }
>> 2017-01-18 12:17:05,985 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-7f0de3b8) (logid:e9bbd63a) Begin cleanup
>> expired async-jobs
>> 2017-01-18 12:17:05,991 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-7f0de3b8) (logid:e9bbd63a) End cleanup
>> expired async-jobs
>> 2017-01-18 12:17:14,798 DEBUG [c.c.a.m.AgentManagerImpl]
>> (AgentManager-Handler-8:null) (logid:) SeqA 3-438475: Processing Seq
>> 3-438475: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
>> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,
>> "_loadInfo":"{\n
>> \"connections\": []\n}","wait":0}}] }
>> 2017-01-18 12:17:14,857 DEBUG [c.c.a.m.AgentManagerImpl]
>> (AgentManager-Handler-8:null) (logid:) SeqA 3-438475: Sending Seq
>> 3-438475: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1, Flags:
>> 100010,
>> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}]
>> }
>> 2017-01-18 12:17:15,984 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-5682fe3c) (logid:901efb52) Begin cleanup
>> expired async-jobs
>> 2017-01-18 12:17:15,990 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-5682fe3c) (logid:901efb52) End cleanup
>> expired async-jobs
>> 2017-01-18 12:17:16,046 DEBUG
>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> (RouterStatusMonitor-1:ctx-7ad0b18a) (logid:b5b19f2b) Found 0 routers
>> to update status.
>> 2017-01-18 12:17:16,047 DEBUG
>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> (RouterStatusMonitor-1:ctx-7ad0b18a) (logid:b5b19f2b) Found 0 VPC
>> networks to update Redundant State.
>> 2017-01-18 12:17:16,049 DEBUG
>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> (RouterStatusMonitor-1:ctx-7ad0b18a) (logid:b5b19f2b) Found 0 networks
>> to update RvR status.
>> 2017-01-18 12:17:16,097 DEBUG
>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> (RouterStatusMonitor-1:ctx-10a8aefd) (logid:b09ffa05) Found 0 routers
>> to update status.
>> 2017-01-18 12:17:16,099 DEBUG
>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> (RouterStatusMonitor-1:ctx-10a8aefd) (logid:b09ffa05) Found 0 VPC
>> networks to update Redundant State.
>> 2017-01-18 12:17:16,101 DEBUG
>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> (RouterStatusMonitor-1:ctx-10a8aefd) (logid:b09ffa05) Found 0 networks
>> to update RvR status.
>> 2017-01-18 12:17:19,799 DEBUG [c.c.a.m.AgentManagerImpl]
>> (AgentManager-Handler-3:null) (logid:) SeqA 3-438476: Processing Seq
>> 3-438476: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
>> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,
>> "_loadInfo":"{\n
>> \"connections\": []\n}","wait":0}}] }
>> 2017-01-18 12:17:19,925 DEBUG [c.c.a.m.AgentManagerImpl]
>> (AgentManager-Handler-3:null) (logid:) SeqA 3-438476: Sending Seq
>> 3-438476: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1, Flags:
>> 100010,
>> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}]
>> }
>> 2017-01-18 12:17:25,985 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-a4c8ceb4) (logid:d0d85ed6) Begin cleanup
>> expired async-jobs
>> 2017-01-18 12:17:25,991 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-a4c8ceb4) (logid:d0d85ed6) End cleanup
>> expired async-jobs
>> 2017-01-18 12:17:29,800 DEBUG [c.c.a.m.AgentManagerImpl]
>> (AgentManager-Handler-7:null) (logid:) SeqA 3-438477: Processing Seq
>> 3-438477: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
>> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,
>> "_loadInfo":"{\n
>> \"connections\": []\n}","wait":0}}] }
>> 2017-01-18 12:17:29,860 DEBUG [c.c.a.m.AgentManagerImpl]
>> (AgentManager-Handler-7:null) (logid:) SeqA 3-438477: Sending Seq
>> 3-438477: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1, Flags:
>> 100010,
>> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}]
>> }
>> 2017-01-18 12:17:30,844 DEBUG [o.a.c.s.SecondaryStorageManagerImpl]
>> (secstorage-1:ctx-f2d92c7a) (logid:6bcbea1f) Zone 1 is ready to launch
>> secondary storage VM
>> 2017-01-18 12:17:31,095 DEBUG [c.c.c.ConsoleProxyManagerImpl]
>> (consoleproxy-1:ctx-5c5c1698) (logid:a199e87c) Zone 1 is ready to
>> launch console proxy
>> 2017-01-18 12:17:34,801 DEBUG [c.c.a.m.AgentManagerImpl]
>> (AgentManager-Handler-4:null) (logid:) SeqA 3-438478: Processing Seq
>> 3-438478: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
>> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,
>> "_loadInfo":"{\n
>> \"connections\": []\n}","wait":0}}] }
>> 2017-01-18 12:17:34,853 DEBUG [c.c.a.m.AgentManagerImpl]
>> (AgentManager-Handler-4:null) (logid:) SeqA 3-438478: Sending Seq
>> 3-438478: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1, Flags:
>> 100010,
>> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}]
>> }
>> 2017-01-18 12:17:35,534 DEBUG [c.c.s.StatsCollector]
>> (StatsCollector-4:ctx-37da523c) (logid:aecb957a) AutoScaling Monitor
>> is running...
>> 2017-01-18 12:17:35,984 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-d031d5da) (logid:722c8ae3) Begin cleanup
>> expired async-jobs
>> 2017-01-18 12:17:35,990 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-d031d5da) (logid:722c8ae3) End cleanup
>> expired async-jobs
>> 2017-01-18 12:17:44,802 DEBUG [c.c.a.m.AgentManagerImpl]
>> (AgentManager-Handler-1:null) (logid:) SeqA 3-438479: Processing Seq
>> 3-438479: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
>> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,
>> "_loadInfo":"{\n
>> \"connections\": []\n}","wait":0}}] }
>> 2017-01-18 12:17:44,863 DEBUG [c.c.a.m.AgentManagerImpl]
>> (AgentManager-Handler-1:null) (logid:) SeqA 3-438479: Sending Seq
>> 3-438479: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1, Flags:
>> 100010,
>> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}]
>> }
>> 2017-01-18 12:17:45,754 DEBUG [c.c.a.m.AgentManagerImpl]
>> (AgentManager-Handler-12:null) (logid:) Ping from 1(iuatc)
>> 2017-01-18 12:17:45,754 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
>> (AgentManager-Handler-12:null) (logid:) Process host VM state report
>> from ping process. host: 1
>> 2017-01-18 12:17:45,760 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
>> (AgentManager-Handler-12:null) (logid:) Process VM state report. host:
>> 1, number of records in report: 2
>> 2017-01-18 12:17:45,760 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
>> (AgentManager-Handler-12:null) (logid:) VM state report. host: 1, vm
>> id:
>> 1, power state: PowerOn
>> 2017-01-18 12:17:45,763 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
>> (AgentManager-Handler-12:null) (logid:) VM power state does not
>> change, skip DB writing. vm id: 1
>> 2017-01-18 12:17:45,763 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
>> (AgentManager-Handler-12:null) (logid:) VM state report. host: 1, vm
>> id:
>> 2, power state: PowerOn
>> 2017-01-18 12:17:45,766 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
>> (AgentManager-Handler-12:null) (logid:) VM power state does not
>> change, skip DB writing. vm id: 2
>> 2017-01-18 12:17:45,769 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
>> (AgentManager-Handler-12:null) (logid:) Done with process of VM state
>> report. host: 1
>> 2017-01-18 12:17:45,985 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-b39e7e84) (logid:d0f90660) Begin cleanup
>> expired async-jobs
>> 2017-01-18 12:17:45,991 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-b39e7e84) (logid:d0f90660) End cleanup
>> expired async-jobs
>> 2017-01-18 12:17:46,046 DEBUG
>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> (RouterStatusMonitor-1:ctx-36fbb5c2) (logid:d346d004) Found 0 routers
>> to update status.
>> 2017-01-18 12:17:46,047 DEBUG
>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> (RouterStatusMonitor-1:ctx-36fbb5c2) (logid:d346d004) Found 0 VPC
>> networks to update Redundant State.
>> 2017-01-18 12:17:46,049 DEBUG
>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> (RouterStatusMonitor-1:ctx-36fbb5c2) (logid:d346d004) Found 0 networks
>> to update RvR status.
>> 2017-01-18 12:17:46,097 DEBUG
>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> (RouterStatusMonitor-1:ctx-16234697) (logid:beb5fec7) Found 0 routers
>> to update status.
>> 2017-01-18 12:17:46,099 DEBUG
>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> (RouterStatusMonitor-1:ctx-16234697) (logid:beb5fec7) Found 0 VPC
>> networks to update Redundant State.
>> 2017-01-18 12:17:46,100 DEBUG
>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> (RouterStatusMonitor-1:ctx-16234697) (logid:beb5fec7) Found 0 networks
>> to update RvR status.
>> 2017-01-18 12:17:46,201 DEBUG [c.c.a.m.AgentManagerImpl]
>> (AgentManager-Handler-2:null) (logid:) Ping from 2(s-2-VM)
>> 2017-01-18 12:17:47,521 DEBUG [c.c.a.m.AgentManagerImpl]
>> (AgentManager-Handler-14:null) (logid:) Ping from 3(v-1-VM)
>> 2017-01-18 12:17:49,842 DEBUG [c.c.a.m.AgentManagerImpl]
>> (AgentManager-Handler-13:null) (logid:) SeqA 3-438481: Processing Seq
>> 3-438481: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
>> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,
>> "_loadInfo":"{\n
>> \"connections\": []\n}","wait":0}}] }
>> 2017-01-18 12:17:49,897 DEBUG [c.c.a.m.AgentManagerImpl]
>> (AgentManager-Handler-13:null) (logid:) SeqA 3-438481: Sending Seq
>> 3-438481: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1, Flags:
>> 100010,
>> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}]
>> }
>> 2017-01-18 12:17:55,985 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-c0db35b5) (logid:c304f172) Begin cleanup
>> expired async-jobs
>> 2017-01-18 12:17:55,991 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-c0db35b5) (logid:c304f172) End cleanup
>> expired async-jobs
>> 2017-01-18 12:17:59,250 DEBUG [c.c.s.StatsCollector]
>> (StatsCollector-6:ctx-eec84206) (logid:b88e0ce8) VmStatsCollector is
>> running...
>> 2017-01-18 12:17:59,803 DEBUG [c.c.a.m.AgentManagerImpl]
>> (AgentManager-Handler-11:null) (logid:) SeqA 3-438482: Processing Seq
>> 3-438482: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
>> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,
>> "_loadInfo":"{\n
>> \"connections\": []\n}","wait":0}}] }
>> 2017-01-18 12:17:59,857 DEBUG [c.c.a.m.AgentManagerImpl]
>> (AgentManager-Handler-11:null) (logid:) SeqA 3-438482: Sending Seq
>> 3-438482: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1, Flags:
>> 100010,
>> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}]
>> }
>> 2017-01-18 12:18:00,843 DEBUG [o.a.c.s.SecondaryStorageManagerImpl]
>> (secstorage-1:ctx-f45be089) (logid:934bd9f7) Zone 1 is ready to launch
>> secondary storage VM
>> 2017-01-18 12:18:01,094 DEBUG [c.c.c.ConsoleProxyManagerImpl]
>> (consoleproxy-1:ctx-4b11dfe0) (logid:cce52dbb) Zone 1 is ready to
>> launch console proxy
>> 2017-01-18 12:18:03,345 DEBUG [c.c.s.StatsCollector]
>> (StatsCollector-2:ctx-c743d4ee) (logid:c1f62e31) HostStatsCollector is
>> running...
>> 2017-01-18 12:18:03,406 DEBUG [c.c.a.t.Request]
>> (StatsCollector-2:ctx-c743d4ee) (logid:c1f62e31) Seq
>> 1-6227070909769882805: Received: { Ans: , MgmtId: 279278805450954,
>> via:
>> 1(iuatc), Ver: v1, Flags: 10, { GetHostStatsAnswer } }
>> 2017-01-18 12:18:04,804 DEBUG [c.c.a.m.AgentManagerImpl]
>> (AgentManager-Handler-9:null) (logid:) SeqA 3-438483: Processing Seq
>> 3-438483: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
>> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,
>> "_loadInfo":"{\n
>> \"connections\": []\n}","wait":0}}] }
>> 2017-01-18 12:18:04,858 DEBUG [c.c.a.m.AgentManagerImpl]
>> (AgentManager-Handler-9:null) (logid:) SeqA 3-438483: Sending Seq
>> 3-438483: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1, Flags:
>> 100010,
>> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}]
>> }
>> 2017-01-18 12:18:05,985 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-149380f2) (logid:53d166f9) Begin cleanup
>> expired async-jobs
>> 2017-01-18 12:18:05,990 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-149380f2) (logid:53d166f9) End cleanup
>> expired async-jobs
>> 2017-01-18 12:18:14,805 DEBUG [c.c.a.m.AgentManagerImpl]
>> (AgentManager-Handler-5:null) (logid:) SeqA 3-438484: Processing Seq
>> 3-438484: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
>> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,
>> "_loadInfo":"{\n
>> \"connections\": []\n}","wait":0}}] }
>> 2017-01-18 12:18:14,951 DEBUG [c.c.a.m.AgentManagerImpl]
>> (AgentManager-Handler-5:null) (logid:) SeqA 3-438484: Sending Seq
>> 3-438484: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1, Flags:
>> 100010,
>> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}]
>> }
>> 2017-01-18 12:18:15,985 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-7ccabf83) (logid:2b0f34c4) Begin cleanup
>> expired async-jobs
>> 2017-01-18 12:18:15,993 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (AsyncJobMgr-Heartbeat-1:ctx-7ccabf83) (logid:2b0f34c4) End cleanup
>> expired async-jobs
>> 2017-01-18 12:18:16,046 DEBUG
>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> (RouterStatusMonitor-1:ctx-fc5408a0) (logid:bbd625c1) Found 0 routers
>> to update status.
>> 2017-01-18 12:18:16,048 DEBUG
>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> (RouterStatusMonitor-1:ctx-fc5408a0) (logid:bbd625c1) Found 0 VPC
>> networks to update Redundant State.
>> 2017-01-18 12:18:16,049 DEBUG
>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> (RouterStatusMonitor-1:ctx-fc5408a0) (logid:bbd625c1) Found 0 networks
>> to update RvR status.
>> 2017-01-18 12:18:16,097 DEBUG
>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> (RouterStatusMonitor-1:ctx-2c697f6b) (logid:cfa955f2) Found 0 routers
>> to update status.
>> 2017-01-18 12:18:16,099 DEBUG
>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> (RouterStatusMonitor-1:ctx-2c697f6b) (logid:cfa955f2) Found 0 VPC
>> networks to update Redundant State.
>> 2017-01-18 12:18:16,101 DEBUG
>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> (RouterStatusMonitor-1:ctx-2c697f6b) (logid:cfa955f2) Found 0 networks
>> to update RvR status.
>> 2017-01-18 12:18:24,389 DEBUG [c.c.h.d.HostDaoImpl]
>> (ClusteredAgentManager
>> Timer:ctx-d699e7ed) (logid:e10673e8) Resetting hosts suitable for
>> reconnect
>> 2017-01-18 12:18:24,392 DEBUG [c.c.h.d.HostDaoImpl]
>> (ClusteredAgentManager
>> Timer:ctx-d699e7ed) (logid:e10673e8) Completed resetting hosts
>> suitable for reconnect
>> 2017-01-18 12:18:24,392 DEBUG [c.c.h.d.HostDaoImpl]
>> (ClusteredAgentManager
>> Timer:ctx-d699e7ed) (logid:e10673e8) Acquiring hosts for clusters
>> already owned by this management server
>> 2017-01-18 12:18:24,392 DEBUG [c.c.h.d.HostDaoImpl]
>> (ClusteredAgentManager
>> Timer:ctx-d699e7ed) (logid:e10673e8) Completed acquiring hosts for
>> clusters already owned by this management server
>> 2017-01-18 12:18:24,392 DEBUG [c.c.h.d.HostDaoImpl]
>> (ClusteredAgentManager
>> Timer:ctx-d699e7ed) (logid:e10673e8) Acquiring hosts for clusters not
>> owned by any management server
>> 2017-01-18 12:18:24,393 DEBUG [c.c.h.d.HostDaoImpl]
>> (ClusteredAgentManager
>> Timer:ctx-d699e7ed) (logid:e10673e8) Completed acquiring hosts for
>> clusters not owned by any management server
>> 2017-01-18 12:18:24,807 DEBUG [c.c.a.m.AgentManagerImpl]
>> (AgentManager-Handler-6:null) (logid:) SeqA 3-438485: Processing Seq
>> 3-438485: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
>> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,
>> "_loadInfo":"{\n
>> \"connections\": []\n}","wait":0}}] }
>> 2017-01-18 12:18:24,870 DEBUG [c.c.a.m.AgentManagerImpl]
>> (AgentManager-Handler-6:null) (logid:) SeqA 3-438485: Sending Seq
>> 3-438485: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1, Flags:
>> 100010, [{"com.cloud.agent.api.AgentControlAnswer":{"result":tru
>>
>>> Can you please share your management_server logs
>>>
>>> On Tue, Jan 17, 2017 at 11:47 PM, ì¡°ë\u0152\u20ací\u02dc\u2022
>>> <ca...@renet.kr> wrote:
>>>
>>>> Hi, all
>>>>
>>>> I am facing secondary storage problem.
>>>>
>>>> I am using NFS Server to use primary and secondary storage.
>>>>
>>>> After rebooting the host,
>>>>
>>>> In Dashboard Cloudstack , it says Secondary Storage 0.00 KB / 0.00
>>>> KB
>>>>
>>>> Primary Storage is okay.
>>>>
>>>>
>>>>
>>>> After all, system VMs are not up. It was working fine before
>>>> rebooting.
>>>>
>>>>
>>>>
>>>> I have tried several times of restart of Management service.
>>>>
>>>>
>>>>
>>>> Any ideas?
>>>>
>>>>
>>>>
>>>> Thanks,
>>>>
>>>>
>>>
>>
>>
>>
>
>
>
RE: Cloudstack 4.7 Secondary Storage problem
Posted by Paul Angus <pa...@shapeblue.com>.
Hi Merlin,
To troubleshoot, try mounting the NFS yourself from the SSVM and get that working.
the likely problem is permissions on the NFS mounts. Either the IP of the SSVM is not allowed or the permissions don't include no_root_squash (or equivalent) the SSVM is probably connecting as 'cloud', so connections have to be mapped to 'root'. Finally the SSVM only reliably supports NFSv3
Kind regards,
Paul Angus
paul.angus@shapeblue.com
www.shapeblue.com
53 Chandos Place, Covent Garden, London WC2N 4HSUK
@shapeblue
-----Original Message-----
From: merlin@students.iitmandi.ac.in [mailto:merlin@students.iitmandi.ac.in]
Sent: 18 January 2017 08:48
To: users@cloudstack.apache.org
Subject: Re: Cloudstack 4.7 Secondary Storage problem
And this is the result of check.sh
2 packets transmitted, 2 packets received, 0% packet loss round-trip min/avg/max/stddev = 0.400/0.550/0.701/0.151 ms
Good: Can ping DNS server
================================================
Good: DNS resolves download.cloud.com
================================================
ERROR: NFS is not currently mounted
Try manually mounting from inside the VM NFS server is 10.8.0.1 PING 10.8.0.1 (10.8.0.1): 48 data bytes
56 bytes from 10.8.0.1: icmp_seq=0 ttl=64 time=0.456 ms
56 bytes from 10.8.0.1: icmp_seq=1 ttl=64 time=0.708 ms
--- 10.8.0.1 ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss round-trip min/avg/max/stddev = 0.456/0.582/0.708/0.126 ms
Good: Can ping nfs server
================================================
Management server is 10.8.19.111. Checking connectivity.
Good: Can connect to management server port 8250 ================================================
Good: Java process is running
================================================
Tests Complete. Look for ERROR or WARNING above.
The problem is that while installation I did mount nfs?
I dont understand what else to do next
> Hi, even I am facing the same problem.. and here are the logs from today:
>
>
>
> 2017-01-18 12:16:46,100 DEBUG
> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-bc5cb4cd) (logid:0ffac658) Found 0 networks
> to update RvR status.
> 2017-01-18 12:16:46,200 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentManager-Handler-11:null) (logid:) Ping from 2(s-2-VM)
> 2017-01-18 12:16:47,521 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentManager-Handler-15:null) (logid:) Ping from 3(v-1-VM)
> 2017-01-18 12:16:49,834 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentManager-Handler-9:null) (logid:) SeqA 3-438472: Processing Seq
> 3-438472: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,
> "_loadInfo":"{\n
> \"connections\": []\n}","wait":0}}] }
> 2017-01-18 12:16:49,887 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentManager-Handler-9:null) (logid:) SeqA 3-438472: Sending Seq
> 3-438472: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1, Flags:
> 100010,
> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}]
> }
> 2017-01-18 12:16:54,390 DEBUG [c.c.h.d.HostDaoImpl]
> (ClusteredAgentManager
> Timer:ctx-8464e4b3) (logid:1f15c721) Resetting hosts suitable for
> reconnect
> 2017-01-18 12:16:54,392 DEBUG [c.c.h.d.HostDaoImpl]
> (ClusteredAgentManager
> Timer:ctx-8464e4b3) (logid:1f15c721) Completed resetting hosts
> suitable for reconnect
> 2017-01-18 12:16:54,392 DEBUG [c.c.h.d.HostDaoImpl]
> (ClusteredAgentManager
> Timer:ctx-8464e4b3) (logid:1f15c721) Acquiring hosts for clusters
> already owned by this management server
> 2017-01-18 12:16:54,393 DEBUG [c.c.h.d.HostDaoImpl]
> (ClusteredAgentManager
> Timer:ctx-8464e4b3) (logid:1f15c721) Completed acquiring hosts for
> clusters already owned by this management server
> 2017-01-18 12:16:54,393 DEBUG [c.c.h.d.HostDaoImpl]
> (ClusteredAgentManager
> Timer:ctx-8464e4b3) (logid:1f15c721) Acquiring hosts for clusters not
> owned by any management server
> 2017-01-18 12:16:54,394 DEBUG [c.c.h.d.HostDaoImpl]
> (ClusteredAgentManager
> Timer:ctx-8464e4b3) (logid:1f15c721) Completed acquiring hosts for
> clusters not owned by any management server
> 2017-01-18 12:16:54,795 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentManager-Handler-5:null) (logid:) SeqA 3-438473: Processing Seq
> 3-438473: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,
> "_loadInfo":"{\n
> \"connections\": []\n}","wait":0}}] }
> 2017-01-18 12:16:54,854 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentManager-Handler-5:null) (logid:) SeqA 3-438473: Sending Seq
> 3-438473: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1, Flags:
> 100010,
> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}]
> }
> 2017-01-18 12:16:55,984 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-d0469453) (logid:af5d4f47) Begin cleanup
> expired async-jobs
> 2017-01-18 12:16:55,991 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-d0469453) (logid:af5d4f47) End cleanup
> expired async-jobs
> 2017-01-18 12:16:59,245 DEBUG [c.c.s.StatsCollector]
> (StatsCollector-1:ctx-68069dff) (logid:a7a6974d) VmStatsCollector is
> running...
> 2017-01-18 12:17:00,841 DEBUG [o.a.c.s.SecondaryStorageManagerImpl]
> (secstorage-1:ctx-4e101aa5) (logid:65f1f0cb) Zone 1 is ready to launch
> secondary storage VM
> 2017-01-18 12:17:01,095 DEBUG [c.c.c.ConsoleProxyManagerImpl]
> (consoleproxy-1:ctx-393828fa) (logid:7b3d0491) Zone 1 is ready to
> launch console proxy
> 2017-01-18 12:17:03,283 DEBUG [c.c.s.StatsCollector]
> (StatsCollector-3:ctx-cc1f58a1) (logid:3ae41340) HostStatsCollector is
> running...
> 2017-01-18 12:17:03,343 DEBUG [c.c.a.t.Request]
> (StatsCollector-3:ctx-cc1f58a1) (logid:3ae41340) Seq
> 1-6227070909769882804: Received: { Ans: , MgmtId: 279278805450954, via:
> 1(iuatc), Ver: v1, Flags: 10, { GetHostStatsAnswer } }
> 2017-01-18 12:17:04,796 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentManager-Handler-10:null) (logid:) SeqA 3-438474: Processing Seq
> 3-438474: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,
> "_loadInfo":"{\n
> \"connections\": []\n}","wait":0}}] }
> 2017-01-18 12:17:04,847 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentManager-Handler-10:null) (logid:) SeqA 3-438474: Sending Seq
> 3-438474: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1, Flags:
> 100010,
> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}]
> }
> 2017-01-18 12:17:05,985 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-7f0de3b8) (logid:e9bbd63a) Begin cleanup
> expired async-jobs
> 2017-01-18 12:17:05,991 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-7f0de3b8) (logid:e9bbd63a) End cleanup
> expired async-jobs
> 2017-01-18 12:17:14,798 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentManager-Handler-8:null) (logid:) SeqA 3-438475: Processing Seq
> 3-438475: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,
> "_loadInfo":"{\n
> \"connections\": []\n}","wait":0}}] }
> 2017-01-18 12:17:14,857 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentManager-Handler-8:null) (logid:) SeqA 3-438475: Sending Seq
> 3-438475: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1, Flags:
> 100010,
> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}]
> }
> 2017-01-18 12:17:15,984 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-5682fe3c) (logid:901efb52) Begin cleanup
> expired async-jobs
> 2017-01-18 12:17:15,990 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-5682fe3c) (logid:901efb52) End cleanup
> expired async-jobs
> 2017-01-18 12:17:16,046 DEBUG
> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-7ad0b18a) (logid:b5b19f2b) Found 0 routers
> to update status.
> 2017-01-18 12:17:16,047 DEBUG
> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-7ad0b18a) (logid:b5b19f2b) Found 0 VPC
> networks to update Redundant State.
> 2017-01-18 12:17:16,049 DEBUG
> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-7ad0b18a) (logid:b5b19f2b) Found 0 networks
> to update RvR status.
> 2017-01-18 12:17:16,097 DEBUG
> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-10a8aefd) (logid:b09ffa05) Found 0 routers
> to update status.
> 2017-01-18 12:17:16,099 DEBUG
> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-10a8aefd) (logid:b09ffa05) Found 0 VPC
> networks to update Redundant State.
> 2017-01-18 12:17:16,101 DEBUG
> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-10a8aefd) (logid:b09ffa05) Found 0 networks
> to update RvR status.
> 2017-01-18 12:17:19,799 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentManager-Handler-3:null) (logid:) SeqA 3-438476: Processing Seq
> 3-438476: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,
> "_loadInfo":"{\n
> \"connections\": []\n}","wait":0}}] }
> 2017-01-18 12:17:19,925 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentManager-Handler-3:null) (logid:) SeqA 3-438476: Sending Seq
> 3-438476: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1, Flags:
> 100010,
> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}]
> }
> 2017-01-18 12:17:25,985 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-a4c8ceb4) (logid:d0d85ed6) Begin cleanup
> expired async-jobs
> 2017-01-18 12:17:25,991 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-a4c8ceb4) (logid:d0d85ed6) End cleanup
> expired async-jobs
> 2017-01-18 12:17:29,800 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentManager-Handler-7:null) (logid:) SeqA 3-438477: Processing Seq
> 3-438477: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,
> "_loadInfo":"{\n
> \"connections\": []\n}","wait":0}}] }
> 2017-01-18 12:17:29,860 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentManager-Handler-7:null) (logid:) SeqA 3-438477: Sending Seq
> 3-438477: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1, Flags:
> 100010,
> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}]
> }
> 2017-01-18 12:17:30,844 DEBUG [o.a.c.s.SecondaryStorageManagerImpl]
> (secstorage-1:ctx-f2d92c7a) (logid:6bcbea1f) Zone 1 is ready to launch
> secondary storage VM
> 2017-01-18 12:17:31,095 DEBUG [c.c.c.ConsoleProxyManagerImpl]
> (consoleproxy-1:ctx-5c5c1698) (logid:a199e87c) Zone 1 is ready to
> launch console proxy
> 2017-01-18 12:17:34,801 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentManager-Handler-4:null) (logid:) SeqA 3-438478: Processing Seq
> 3-438478: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,
> "_loadInfo":"{\n
> \"connections\": []\n}","wait":0}}] }
> 2017-01-18 12:17:34,853 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentManager-Handler-4:null) (logid:) SeqA 3-438478: Sending Seq
> 3-438478: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1, Flags:
> 100010,
> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}]
> }
> 2017-01-18 12:17:35,534 DEBUG [c.c.s.StatsCollector]
> (StatsCollector-4:ctx-37da523c) (logid:aecb957a) AutoScaling Monitor
> is running...
> 2017-01-18 12:17:35,984 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-d031d5da) (logid:722c8ae3) Begin cleanup
> expired async-jobs
> 2017-01-18 12:17:35,990 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-d031d5da) (logid:722c8ae3) End cleanup
> expired async-jobs
> 2017-01-18 12:17:44,802 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentManager-Handler-1:null) (logid:) SeqA 3-438479: Processing Seq
> 3-438479: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,
> "_loadInfo":"{\n
> \"connections\": []\n}","wait":0}}] }
> 2017-01-18 12:17:44,863 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentManager-Handler-1:null) (logid:) SeqA 3-438479: Sending Seq
> 3-438479: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1, Flags:
> 100010,
> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}]
> }
> 2017-01-18 12:17:45,754 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentManager-Handler-12:null) (logid:) Ping from 1(iuatc)
> 2017-01-18 12:17:45,754 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> (AgentManager-Handler-12:null) (logid:) Process host VM state report
> from ping process. host: 1
> 2017-01-18 12:17:45,760 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> (AgentManager-Handler-12:null) (logid:) Process VM state report. host:
> 1, number of records in report: 2
> 2017-01-18 12:17:45,760 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> (AgentManager-Handler-12:null) (logid:) VM state report. host: 1, vm id:
> 1, power state: PowerOn
> 2017-01-18 12:17:45,763 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> (AgentManager-Handler-12:null) (logid:) VM power state does not
> change, skip DB writing. vm id: 1
> 2017-01-18 12:17:45,763 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> (AgentManager-Handler-12:null) (logid:) VM state report. host: 1, vm id:
> 2, power state: PowerOn
> 2017-01-18 12:17:45,766 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> (AgentManager-Handler-12:null) (logid:) VM power state does not
> change, skip DB writing. vm id: 2
> 2017-01-18 12:17:45,769 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> (AgentManager-Handler-12:null) (logid:) Done with process of VM state
> report. host: 1
> 2017-01-18 12:17:45,985 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-b39e7e84) (logid:d0f90660) Begin cleanup
> expired async-jobs
> 2017-01-18 12:17:45,991 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-b39e7e84) (logid:d0f90660) End cleanup
> expired async-jobs
> 2017-01-18 12:17:46,046 DEBUG
> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-36fbb5c2) (logid:d346d004) Found 0 routers
> to update status.
> 2017-01-18 12:17:46,047 DEBUG
> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-36fbb5c2) (logid:d346d004) Found 0 VPC
> networks to update Redundant State.
> 2017-01-18 12:17:46,049 DEBUG
> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-36fbb5c2) (logid:d346d004) Found 0 networks
> to update RvR status.
> 2017-01-18 12:17:46,097 DEBUG
> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-16234697) (logid:beb5fec7) Found 0 routers
> to update status.
> 2017-01-18 12:17:46,099 DEBUG
> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-16234697) (logid:beb5fec7) Found 0 VPC
> networks to update Redundant State.
> 2017-01-18 12:17:46,100 DEBUG
> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-16234697) (logid:beb5fec7) Found 0 networks
> to update RvR status.
> 2017-01-18 12:17:46,201 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentManager-Handler-2:null) (logid:) Ping from 2(s-2-VM)
> 2017-01-18 12:17:47,521 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentManager-Handler-14:null) (logid:) Ping from 3(v-1-VM)
> 2017-01-18 12:17:49,842 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentManager-Handler-13:null) (logid:) SeqA 3-438481: Processing Seq
> 3-438481: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,
> "_loadInfo":"{\n
> \"connections\": []\n}","wait":0}}] }
> 2017-01-18 12:17:49,897 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentManager-Handler-13:null) (logid:) SeqA 3-438481: Sending Seq
> 3-438481: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1, Flags:
> 100010,
> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}]
> }
> 2017-01-18 12:17:55,985 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-c0db35b5) (logid:c304f172) Begin cleanup
> expired async-jobs
> 2017-01-18 12:17:55,991 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-c0db35b5) (logid:c304f172) End cleanup
> expired async-jobs
> 2017-01-18 12:17:59,250 DEBUG [c.c.s.StatsCollector]
> (StatsCollector-6:ctx-eec84206) (logid:b88e0ce8) VmStatsCollector is
> running...
> 2017-01-18 12:17:59,803 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentManager-Handler-11:null) (logid:) SeqA 3-438482: Processing Seq
> 3-438482: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,
> "_loadInfo":"{\n
> \"connections\": []\n}","wait":0}}] }
> 2017-01-18 12:17:59,857 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentManager-Handler-11:null) (logid:) SeqA 3-438482: Sending Seq
> 3-438482: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1, Flags:
> 100010,
> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}]
> }
> 2017-01-18 12:18:00,843 DEBUG [o.a.c.s.SecondaryStorageManagerImpl]
> (secstorage-1:ctx-f45be089) (logid:934bd9f7) Zone 1 is ready to launch
> secondary storage VM
> 2017-01-18 12:18:01,094 DEBUG [c.c.c.ConsoleProxyManagerImpl]
> (consoleproxy-1:ctx-4b11dfe0) (logid:cce52dbb) Zone 1 is ready to
> launch console proxy
> 2017-01-18 12:18:03,345 DEBUG [c.c.s.StatsCollector]
> (StatsCollector-2:ctx-c743d4ee) (logid:c1f62e31) HostStatsCollector is
> running...
> 2017-01-18 12:18:03,406 DEBUG [c.c.a.t.Request]
> (StatsCollector-2:ctx-c743d4ee) (logid:c1f62e31) Seq
> 1-6227070909769882805: Received: { Ans: , MgmtId: 279278805450954, via:
> 1(iuatc), Ver: v1, Flags: 10, { GetHostStatsAnswer } }
> 2017-01-18 12:18:04,804 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentManager-Handler-9:null) (logid:) SeqA 3-438483: Processing Seq
> 3-438483: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,
> "_loadInfo":"{\n
> \"connections\": []\n}","wait":0}}] }
> 2017-01-18 12:18:04,858 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentManager-Handler-9:null) (logid:) SeqA 3-438483: Sending Seq
> 3-438483: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1, Flags:
> 100010,
> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}]
> }
> 2017-01-18 12:18:05,985 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-149380f2) (logid:53d166f9) Begin cleanup
> expired async-jobs
> 2017-01-18 12:18:05,990 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-149380f2) (logid:53d166f9) End cleanup
> expired async-jobs
> 2017-01-18 12:18:14,805 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentManager-Handler-5:null) (logid:) SeqA 3-438484: Processing Seq
> 3-438484: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,
> "_loadInfo":"{\n
> \"connections\": []\n}","wait":0}}] }
> 2017-01-18 12:18:14,951 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentManager-Handler-5:null) (logid:) SeqA 3-438484: Sending Seq
> 3-438484: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1, Flags:
> 100010,
> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}]
> }
> 2017-01-18 12:18:15,985 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-7ccabf83) (logid:2b0f34c4) Begin cleanup
> expired async-jobs
> 2017-01-18 12:18:15,993 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-7ccabf83) (logid:2b0f34c4) End cleanup
> expired async-jobs
> 2017-01-18 12:18:16,046 DEBUG
> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-fc5408a0) (logid:bbd625c1) Found 0 routers
> to update status.
> 2017-01-18 12:18:16,048 DEBUG
> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-fc5408a0) (logid:bbd625c1) Found 0 VPC
> networks to update Redundant State.
> 2017-01-18 12:18:16,049 DEBUG
> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-fc5408a0) (logid:bbd625c1) Found 0 networks
> to update RvR status.
> 2017-01-18 12:18:16,097 DEBUG
> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-2c697f6b) (logid:cfa955f2) Found 0 routers
> to update status.
> 2017-01-18 12:18:16,099 DEBUG
> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-2c697f6b) (logid:cfa955f2) Found 0 VPC
> networks to update Redundant State.
> 2017-01-18 12:18:16,101 DEBUG
> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-2c697f6b) (logid:cfa955f2) Found 0 networks
> to update RvR status.
> 2017-01-18 12:18:24,389 DEBUG [c.c.h.d.HostDaoImpl]
> (ClusteredAgentManager
> Timer:ctx-d699e7ed) (logid:e10673e8) Resetting hosts suitable for
> reconnect
> 2017-01-18 12:18:24,392 DEBUG [c.c.h.d.HostDaoImpl]
> (ClusteredAgentManager
> Timer:ctx-d699e7ed) (logid:e10673e8) Completed resetting hosts
> suitable for reconnect
> 2017-01-18 12:18:24,392 DEBUG [c.c.h.d.HostDaoImpl]
> (ClusteredAgentManager
> Timer:ctx-d699e7ed) (logid:e10673e8) Acquiring hosts for clusters
> already owned by this management server
> 2017-01-18 12:18:24,392 DEBUG [c.c.h.d.HostDaoImpl]
> (ClusteredAgentManager
> Timer:ctx-d699e7ed) (logid:e10673e8) Completed acquiring hosts for
> clusters already owned by this management server
> 2017-01-18 12:18:24,392 DEBUG [c.c.h.d.HostDaoImpl]
> (ClusteredAgentManager
> Timer:ctx-d699e7ed) (logid:e10673e8) Acquiring hosts for clusters not
> owned by any management server
> 2017-01-18 12:18:24,393 DEBUG [c.c.h.d.HostDaoImpl]
> (ClusteredAgentManager
> Timer:ctx-d699e7ed) (logid:e10673e8) Completed acquiring hosts for
> clusters not owned by any management server
> 2017-01-18 12:18:24,807 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentManager-Handler-6:null) (logid:) SeqA 3-438485: Processing Seq
> 3-438485: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,
> "_loadInfo":"{\n
> \"connections\": []\n}","wait":0}}] }
> 2017-01-18 12:18:24,870 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentManager-Handler-6:null) (logid:) SeqA 3-438485: Sending Seq
> 3-438485: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1, Flags:
> 100010, [{"com.cloud.agent.api.AgentControlAnswer":{"result":tru
>
>> Can you please share your management_server logs
>>
>> On Tue, Jan 17, 2017 at 11:47 PM, 조대형 <ca...@renet.kr> wrote:
>>
>>> Hi, all
>>>
>>> I am facing secondary storage problem.
>>>
>>> I am using NFS Server to use primary and secondary storage.
>>>
>>> After rebooting the host,
>>>
>>> In Dashboard Cloudstack , it says Secondary Storage 0.00 KB / 0.00
>>> KB
>>>
>>> Primary Storage is okay.
>>>
>>>
>>>
>>> After all, system VMs are not up. It was working fine before rebooting.
>>>
>>>
>>>
>>> I have tried several times of restart of Management service.
>>>
>>>
>>>
>>> Any ideas?
>>>
>>>
>>>
>>> Thanks,
>>>
>>>
>>
>
>
>
Re: Cloudstack 4.7 Secondary Storage problem
Posted by me...@students.iitmandi.ac.in.
And this is the result of check.sh
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max/stddev = 0.400/0.550/0.701/0.151 ms
Good: Can ping DNS server
================================================
Good: DNS resolves download.cloud.com
================================================
ERROR: NFS is not currently mounted
Try manually mounting from inside the VM
NFS server is 10.8.0.1
PING 10.8.0.1 (10.8.0.1): 48 data bytes
56 bytes from 10.8.0.1: icmp_seq=0 ttl=64 time=0.456 ms
56 bytes from 10.8.0.1: icmp_seq=1 ttl=64 time=0.708 ms
--- 10.8.0.1 ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max/stddev = 0.456/0.582/0.708/0.126 ms
Good: Can ping nfs server
================================================
Management server is 10.8.19.111. Checking connectivity.
Good: Can connect to management server port 8250
================================================
Good: Java process is running
================================================
Tests Complete. Look for ERROR or WARNING above.
The problem is that while installation I did mount nfs?
I dont understand what else to do next
> Hi, even I am facing the same problem.. and here are the logs from today:
>
>
>
> 2017-01-18 12:16:46,100 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-bc5cb4cd) (logid:0ffac658) Found 0 networks to
> update RvR status.
> 2017-01-18 12:16:46,200 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentManager-Handler-11:null) (logid:) Ping from 2(s-2-VM)
> 2017-01-18 12:16:47,521 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentManager-Handler-15:null) (logid:) Ping from 3(v-1-VM)
> 2017-01-18 12:16:49,834 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentManager-Handler-9:null) (logid:) SeqA 3-438472: Processing Seq
> 3-438472: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,"_loadInfo":"{\n
> \"connections\": []\n}","wait":0}}] }
> 2017-01-18 12:16:49,887 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentManager-Handler-9:null) (logid:) SeqA 3-438472: Sending Seq
> 3-438472: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1, Flags:
> 100010,
> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}] }
> 2017-01-18 12:16:54,390 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
> Timer:ctx-8464e4b3) (logid:1f15c721) Resetting hosts suitable for
> reconnect
> 2017-01-18 12:16:54,392 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
> Timer:ctx-8464e4b3) (logid:1f15c721) Completed resetting hosts suitable
> for reconnect
> 2017-01-18 12:16:54,392 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
> Timer:ctx-8464e4b3) (logid:1f15c721) Acquiring hosts for clusters already
> owned by this management server
> 2017-01-18 12:16:54,393 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
> Timer:ctx-8464e4b3) (logid:1f15c721) Completed acquiring hosts for
> clusters already owned by this management server
> 2017-01-18 12:16:54,393 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
> Timer:ctx-8464e4b3) (logid:1f15c721) Acquiring hosts for clusters not
> owned by any management server
> 2017-01-18 12:16:54,394 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
> Timer:ctx-8464e4b3) (logid:1f15c721) Completed acquiring hosts for
> clusters not owned by any management server
> 2017-01-18 12:16:54,795 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentManager-Handler-5:null) (logid:) SeqA 3-438473: Processing Seq
> 3-438473: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,"_loadInfo":"{\n
> \"connections\": []\n}","wait":0}}] }
> 2017-01-18 12:16:54,854 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentManager-Handler-5:null) (logid:) SeqA 3-438473: Sending Seq
> 3-438473: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1, Flags:
> 100010,
> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}] }
> 2017-01-18 12:16:55,984 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-d0469453) (logid:af5d4f47) Begin cleanup
> expired async-jobs
> 2017-01-18 12:16:55,991 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-d0469453) (logid:af5d4f47) End cleanup
> expired async-jobs
> 2017-01-18 12:16:59,245 DEBUG [c.c.s.StatsCollector]
> (StatsCollector-1:ctx-68069dff) (logid:a7a6974d) VmStatsCollector is
> running...
> 2017-01-18 12:17:00,841 DEBUG [o.a.c.s.SecondaryStorageManagerImpl]
> (secstorage-1:ctx-4e101aa5) (logid:65f1f0cb) Zone 1 is ready to launch
> secondary storage VM
> 2017-01-18 12:17:01,095 DEBUG [c.c.c.ConsoleProxyManagerImpl]
> (consoleproxy-1:ctx-393828fa) (logid:7b3d0491) Zone 1 is ready to launch
> console proxy
> 2017-01-18 12:17:03,283 DEBUG [c.c.s.StatsCollector]
> (StatsCollector-3:ctx-cc1f58a1) (logid:3ae41340) HostStatsCollector is
> running...
> 2017-01-18 12:17:03,343 DEBUG [c.c.a.t.Request]
> (StatsCollector-3:ctx-cc1f58a1) (logid:3ae41340) Seq
> 1-6227070909769882804: Received: { Ans: , MgmtId: 279278805450954, via:
> 1(iuatc), Ver: v1, Flags: 10, { GetHostStatsAnswer } }
> 2017-01-18 12:17:04,796 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentManager-Handler-10:null) (logid:) SeqA 3-438474: Processing Seq
> 3-438474: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,"_loadInfo":"{\n
> \"connections\": []\n}","wait":0}}] }
> 2017-01-18 12:17:04,847 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentManager-Handler-10:null) (logid:) SeqA 3-438474: Sending Seq
> 3-438474: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1, Flags:
> 100010,
> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}] }
> 2017-01-18 12:17:05,985 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-7f0de3b8) (logid:e9bbd63a) Begin cleanup
> expired async-jobs
> 2017-01-18 12:17:05,991 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-7f0de3b8) (logid:e9bbd63a) End cleanup
> expired async-jobs
> 2017-01-18 12:17:14,798 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentManager-Handler-8:null) (logid:) SeqA 3-438475: Processing Seq
> 3-438475: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,"_loadInfo":"{\n
> \"connections\": []\n}","wait":0}}] }
> 2017-01-18 12:17:14,857 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentManager-Handler-8:null) (logid:) SeqA 3-438475: Sending Seq
> 3-438475: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1, Flags:
> 100010,
> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}] }
> 2017-01-18 12:17:15,984 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-5682fe3c) (logid:901efb52) Begin cleanup
> expired async-jobs
> 2017-01-18 12:17:15,990 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-5682fe3c) (logid:901efb52) End cleanup
> expired async-jobs
> 2017-01-18 12:17:16,046 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-7ad0b18a) (logid:b5b19f2b) Found 0 routers to
> update status.
> 2017-01-18 12:17:16,047 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-7ad0b18a) (logid:b5b19f2b) Found 0 VPC networks
> to update Redundant State.
> 2017-01-18 12:17:16,049 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-7ad0b18a) (logid:b5b19f2b) Found 0 networks to
> update RvR status.
> 2017-01-18 12:17:16,097 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-10a8aefd) (logid:b09ffa05) Found 0 routers to
> update status.
> 2017-01-18 12:17:16,099 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-10a8aefd) (logid:b09ffa05) Found 0 VPC networks
> to update Redundant State.
> 2017-01-18 12:17:16,101 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-10a8aefd) (logid:b09ffa05) Found 0 networks to
> update RvR status.
> 2017-01-18 12:17:19,799 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentManager-Handler-3:null) (logid:) SeqA 3-438476: Processing Seq
> 3-438476: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,"_loadInfo":"{\n
> \"connections\": []\n}","wait":0}}] }
> 2017-01-18 12:17:19,925 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentManager-Handler-3:null) (logid:) SeqA 3-438476: Sending Seq
> 3-438476: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1, Flags:
> 100010,
> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}] }
> 2017-01-18 12:17:25,985 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-a4c8ceb4) (logid:d0d85ed6) Begin cleanup
> expired async-jobs
> 2017-01-18 12:17:25,991 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-a4c8ceb4) (logid:d0d85ed6) End cleanup
> expired async-jobs
> 2017-01-18 12:17:29,800 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentManager-Handler-7:null) (logid:) SeqA 3-438477: Processing Seq
> 3-438477: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,"_loadInfo":"{\n
> \"connections\": []\n}","wait":0}}] }
> 2017-01-18 12:17:29,860 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentManager-Handler-7:null) (logid:) SeqA 3-438477: Sending Seq
> 3-438477: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1, Flags:
> 100010,
> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}] }
> 2017-01-18 12:17:30,844 DEBUG [o.a.c.s.SecondaryStorageManagerImpl]
> (secstorage-1:ctx-f2d92c7a) (logid:6bcbea1f) Zone 1 is ready to launch
> secondary storage VM
> 2017-01-18 12:17:31,095 DEBUG [c.c.c.ConsoleProxyManagerImpl]
> (consoleproxy-1:ctx-5c5c1698) (logid:a199e87c) Zone 1 is ready to launch
> console proxy
> 2017-01-18 12:17:34,801 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentManager-Handler-4:null) (logid:) SeqA 3-438478: Processing Seq
> 3-438478: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,"_loadInfo":"{\n
> \"connections\": []\n}","wait":0}}] }
> 2017-01-18 12:17:34,853 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentManager-Handler-4:null) (logid:) SeqA 3-438478: Sending Seq
> 3-438478: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1, Flags:
> 100010,
> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}] }
> 2017-01-18 12:17:35,534 DEBUG [c.c.s.StatsCollector]
> (StatsCollector-4:ctx-37da523c) (logid:aecb957a) AutoScaling Monitor is
> running...
> 2017-01-18 12:17:35,984 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-d031d5da) (logid:722c8ae3) Begin cleanup
> expired async-jobs
> 2017-01-18 12:17:35,990 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-d031d5da) (logid:722c8ae3) End cleanup
> expired async-jobs
> 2017-01-18 12:17:44,802 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentManager-Handler-1:null) (logid:) SeqA 3-438479: Processing Seq
> 3-438479: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,"_loadInfo":"{\n
> \"connections\": []\n}","wait":0}}] }
> 2017-01-18 12:17:44,863 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentManager-Handler-1:null) (logid:) SeqA 3-438479: Sending Seq
> 3-438479: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1, Flags:
> 100010,
> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}] }
> 2017-01-18 12:17:45,754 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentManager-Handler-12:null) (logid:) Ping from 1(iuatc)
> 2017-01-18 12:17:45,754 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> (AgentManager-Handler-12:null) (logid:) Process host VM state report from
> ping process. host: 1
> 2017-01-18 12:17:45,760 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> (AgentManager-Handler-12:null) (logid:) Process VM state report. host: 1,
> number of records in report: 2
> 2017-01-18 12:17:45,760 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> (AgentManager-Handler-12:null) (logid:) VM state report. host: 1, vm id:
> 1, power state: PowerOn
> 2017-01-18 12:17:45,763 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> (AgentManager-Handler-12:null) (logid:) VM power state does not change,
> skip DB writing. vm id: 1
> 2017-01-18 12:17:45,763 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> (AgentManager-Handler-12:null) (logid:) VM state report. host: 1, vm id:
> 2, power state: PowerOn
> 2017-01-18 12:17:45,766 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> (AgentManager-Handler-12:null) (logid:) VM power state does not change,
> skip DB writing. vm id: 2
> 2017-01-18 12:17:45,769 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
> (AgentManager-Handler-12:null) (logid:) Done with process of VM state
> report. host: 1
> 2017-01-18 12:17:45,985 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-b39e7e84) (logid:d0f90660) Begin cleanup
> expired async-jobs
> 2017-01-18 12:17:45,991 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-b39e7e84) (logid:d0f90660) End cleanup
> expired async-jobs
> 2017-01-18 12:17:46,046 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-36fbb5c2) (logid:d346d004) Found 0 routers to
> update status.
> 2017-01-18 12:17:46,047 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-36fbb5c2) (logid:d346d004) Found 0 VPC networks
> to update Redundant State.
> 2017-01-18 12:17:46,049 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-36fbb5c2) (logid:d346d004) Found 0 networks to
> update RvR status.
> 2017-01-18 12:17:46,097 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-16234697) (logid:beb5fec7) Found 0 routers to
> update status.
> 2017-01-18 12:17:46,099 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-16234697) (logid:beb5fec7) Found 0 VPC networks
> to update Redundant State.
> 2017-01-18 12:17:46,100 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-16234697) (logid:beb5fec7) Found 0 networks to
> update RvR status.
> 2017-01-18 12:17:46,201 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentManager-Handler-2:null) (logid:) Ping from 2(s-2-VM)
> 2017-01-18 12:17:47,521 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentManager-Handler-14:null) (logid:) Ping from 3(v-1-VM)
> 2017-01-18 12:17:49,842 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentManager-Handler-13:null) (logid:) SeqA 3-438481: Processing Seq
> 3-438481: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,"_loadInfo":"{\n
> \"connections\": []\n}","wait":0}}] }
> 2017-01-18 12:17:49,897 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentManager-Handler-13:null) (logid:) SeqA 3-438481: Sending Seq
> 3-438481: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1, Flags:
> 100010,
> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}] }
> 2017-01-18 12:17:55,985 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-c0db35b5) (logid:c304f172) Begin cleanup
> expired async-jobs
> 2017-01-18 12:17:55,991 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-c0db35b5) (logid:c304f172) End cleanup
> expired async-jobs
> 2017-01-18 12:17:59,250 DEBUG [c.c.s.StatsCollector]
> (StatsCollector-6:ctx-eec84206) (logid:b88e0ce8) VmStatsCollector is
> running...
> 2017-01-18 12:17:59,803 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentManager-Handler-11:null) (logid:) SeqA 3-438482: Processing Seq
> 3-438482: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,"_loadInfo":"{\n
> \"connections\": []\n}","wait":0}}] }
> 2017-01-18 12:17:59,857 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentManager-Handler-11:null) (logid:) SeqA 3-438482: Sending Seq
> 3-438482: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1, Flags:
> 100010,
> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}] }
> 2017-01-18 12:18:00,843 DEBUG [o.a.c.s.SecondaryStorageManagerImpl]
> (secstorage-1:ctx-f45be089) (logid:934bd9f7) Zone 1 is ready to launch
> secondary storage VM
> 2017-01-18 12:18:01,094 DEBUG [c.c.c.ConsoleProxyManagerImpl]
> (consoleproxy-1:ctx-4b11dfe0) (logid:cce52dbb) Zone 1 is ready to launch
> console proxy
> 2017-01-18 12:18:03,345 DEBUG [c.c.s.StatsCollector]
> (StatsCollector-2:ctx-c743d4ee) (logid:c1f62e31) HostStatsCollector is
> running...
> 2017-01-18 12:18:03,406 DEBUG [c.c.a.t.Request]
> (StatsCollector-2:ctx-c743d4ee) (logid:c1f62e31) Seq
> 1-6227070909769882805: Received: { Ans: , MgmtId: 279278805450954, via:
> 1(iuatc), Ver: v1, Flags: 10, { GetHostStatsAnswer } }
> 2017-01-18 12:18:04,804 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentManager-Handler-9:null) (logid:) SeqA 3-438483: Processing Seq
> 3-438483: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,"_loadInfo":"{\n
> \"connections\": []\n}","wait":0}}] }
> 2017-01-18 12:18:04,858 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentManager-Handler-9:null) (logid:) SeqA 3-438483: Sending Seq
> 3-438483: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1, Flags:
> 100010,
> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}] }
> 2017-01-18 12:18:05,985 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-149380f2) (logid:53d166f9) Begin cleanup
> expired async-jobs
> 2017-01-18 12:18:05,990 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-149380f2) (logid:53d166f9) End cleanup
> expired async-jobs
> 2017-01-18 12:18:14,805 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentManager-Handler-5:null) (logid:) SeqA 3-438484: Processing Seq
> 3-438484: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,"_loadInfo":"{\n
> \"connections\": []\n}","wait":0}}] }
> 2017-01-18 12:18:14,951 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentManager-Handler-5:null) (logid:) SeqA 3-438484: Sending Seq
> 3-438484: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1, Flags:
> 100010,
> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}] }
> 2017-01-18 12:18:15,985 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-7ccabf83) (logid:2b0f34c4) Begin cleanup
> expired async-jobs
> 2017-01-18 12:18:15,993 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-7ccabf83) (logid:2b0f34c4) End cleanup
> expired async-jobs
> 2017-01-18 12:18:16,046 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-fc5408a0) (logid:bbd625c1) Found 0 routers to
> update status.
> 2017-01-18 12:18:16,048 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-fc5408a0) (logid:bbd625c1) Found 0 VPC networks
> to update Redundant State.
> 2017-01-18 12:18:16,049 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-fc5408a0) (logid:bbd625c1) Found 0 networks to
> update RvR status.
> 2017-01-18 12:18:16,097 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-2c697f6b) (logid:cfa955f2) Found 0 routers to
> update status.
> 2017-01-18 12:18:16,099 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-2c697f6b) (logid:cfa955f2) Found 0 VPC networks
> to update Redundant State.
> 2017-01-18 12:18:16,101 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-2c697f6b) (logid:cfa955f2) Found 0 networks to
> update RvR status.
> 2017-01-18 12:18:24,389 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
> Timer:ctx-d699e7ed) (logid:e10673e8) Resetting hosts suitable for
> reconnect
> 2017-01-18 12:18:24,392 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
> Timer:ctx-d699e7ed) (logid:e10673e8) Completed resetting hosts suitable
> for reconnect
> 2017-01-18 12:18:24,392 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
> Timer:ctx-d699e7ed) (logid:e10673e8) Acquiring hosts for clusters already
> owned by this management server
> 2017-01-18 12:18:24,392 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
> Timer:ctx-d699e7ed) (logid:e10673e8) Completed acquiring hosts for
> clusters already owned by this management server
> 2017-01-18 12:18:24,392 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
> Timer:ctx-d699e7ed) (logid:e10673e8) Acquiring hosts for clusters not
> owned by any management server
> 2017-01-18 12:18:24,393 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
> Timer:ctx-d699e7ed) (logid:e10673e8) Completed acquiring hosts for
> clusters not owned by any management server
> 2017-01-18 12:18:24,807 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentManager-Handler-6:null) (logid:) SeqA 3-438485: Processing Seq
> 3-438485: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,"_loadInfo":"{\n
> \"connections\": []\n}","wait":0}}] }
> 2017-01-18 12:18:24,870 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentManager-Handler-6:null) (logid:) SeqA 3-438485: Sending Seq
> 3-438485: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1, Flags:
> 100010, [{"com.cloud.agent.api.AgentControlAnswer":{"result":tru
>
>> Can you please share your management_server logs
>>
>> On Tue, Jan 17, 2017 at 11:47 PM, \uc870\ub300\ud615 <ca...@renet.kr> wrote:
>>
>>> Hi, all
>>>
>>> I am facing secondary storage problem.
>>>
>>> I am using NFS Server to use primary and secondary storage.
>>>
>>> After rebooting the host,
>>>
>>> In Dashboard Cloudstack , it says Secondary Storage 0.00 KB / 0.00 KB
>>>
>>> Primary Storage is okay.
>>>
>>>
>>>
>>> After all, system VMs are not up. It was working fine before rebooting.
>>>
>>>
>>>
>>> I have tried several times of restart of Management service.
>>>
>>>
>>>
>>> Any ideas?
>>>
>>>
>>>
>>> Thanks,
>>>
>>>
>>
>
>
>
Re: Cloudstack 4.7 Secondary Storage problem
Posted by me...@students.iitmandi.ac.in.
Hi, even I am facing the same problem.. and here are the logs from today:
2017-01-18 12:16:46,100 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
(RouterStatusMonitor-1:ctx-bc5cb4cd) (logid:0ffac658) Found 0 networks to
update RvR status.
2017-01-18 12:16:46,200 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentManager-Handler-11:null) (logid:) Ping from 2(s-2-VM)
2017-01-18 12:16:47,521 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentManager-Handler-15:null) (logid:) Ping from 3(v-1-VM)
2017-01-18 12:16:49,834 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentManager-Handler-9:null) (logid:) SeqA 3-438472: Processing Seq
3-438472: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
[{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,"_loadInfo":"{\n
\"connections\": []\n}","wait":0}}] }
2017-01-18 12:16:49,887 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentManager-Handler-9:null) (logid:) SeqA 3-438472: Sending Seq
3-438472: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1, Flags:
100010,
[{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}] }
2017-01-18 12:16:54,390 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
Timer:ctx-8464e4b3) (logid:1f15c721) Resetting hosts suitable for
reconnect
2017-01-18 12:16:54,392 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
Timer:ctx-8464e4b3) (logid:1f15c721) Completed resetting hosts suitable
for reconnect
2017-01-18 12:16:54,392 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
Timer:ctx-8464e4b3) (logid:1f15c721) Acquiring hosts for clusters already
owned by this management server
2017-01-18 12:16:54,393 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
Timer:ctx-8464e4b3) (logid:1f15c721) Completed acquiring hosts for
clusters already owned by this management server
2017-01-18 12:16:54,393 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
Timer:ctx-8464e4b3) (logid:1f15c721) Acquiring hosts for clusters not
owned by any management server
2017-01-18 12:16:54,394 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
Timer:ctx-8464e4b3) (logid:1f15c721) Completed acquiring hosts for
clusters not owned by any management server
2017-01-18 12:16:54,795 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentManager-Handler-5:null) (logid:) SeqA 3-438473: Processing Seq
3-438473: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
[{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,"_loadInfo":"{\n
\"connections\": []\n}","wait":0}}] }
2017-01-18 12:16:54,854 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentManager-Handler-5:null) (logid:) SeqA 3-438473: Sending Seq
3-438473: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1, Flags:
100010,
[{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}] }
2017-01-18 12:16:55,984 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-d0469453) (logid:af5d4f47) Begin cleanup
expired async-jobs
2017-01-18 12:16:55,991 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-d0469453) (logid:af5d4f47) End cleanup
expired async-jobs
2017-01-18 12:16:59,245 DEBUG [c.c.s.StatsCollector]
(StatsCollector-1:ctx-68069dff) (logid:a7a6974d) VmStatsCollector is
running...
2017-01-18 12:17:00,841 DEBUG [o.a.c.s.SecondaryStorageManagerImpl]
(secstorage-1:ctx-4e101aa5) (logid:65f1f0cb) Zone 1 is ready to launch
secondary storage VM
2017-01-18 12:17:01,095 DEBUG [c.c.c.ConsoleProxyManagerImpl]
(consoleproxy-1:ctx-393828fa) (logid:7b3d0491) Zone 1 is ready to launch
console proxy
2017-01-18 12:17:03,283 DEBUG [c.c.s.StatsCollector]
(StatsCollector-3:ctx-cc1f58a1) (logid:3ae41340) HostStatsCollector is
running...
2017-01-18 12:17:03,343 DEBUG [c.c.a.t.Request]
(StatsCollector-3:ctx-cc1f58a1) (logid:3ae41340) Seq
1-6227070909769882804: Received: { Ans: , MgmtId: 279278805450954, via:
1(iuatc), Ver: v1, Flags: 10, { GetHostStatsAnswer } }
2017-01-18 12:17:04,796 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentManager-Handler-10:null) (logid:) SeqA 3-438474: Processing Seq
3-438474: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
[{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,"_loadInfo":"{\n
\"connections\": []\n}","wait":0}}] }
2017-01-18 12:17:04,847 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentManager-Handler-10:null) (logid:) SeqA 3-438474: Sending Seq
3-438474: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1, Flags:
100010,
[{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}] }
2017-01-18 12:17:05,985 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-7f0de3b8) (logid:e9bbd63a) Begin cleanup
expired async-jobs
2017-01-18 12:17:05,991 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-7f0de3b8) (logid:e9bbd63a) End cleanup
expired async-jobs
2017-01-18 12:17:14,798 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentManager-Handler-8:null) (logid:) SeqA 3-438475: Processing Seq
3-438475: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
[{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,"_loadInfo":"{\n
\"connections\": []\n}","wait":0}}] }
2017-01-18 12:17:14,857 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentManager-Handler-8:null) (logid:) SeqA 3-438475: Sending Seq
3-438475: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1, Flags:
100010,
[{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}] }
2017-01-18 12:17:15,984 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-5682fe3c) (logid:901efb52) Begin cleanup
expired async-jobs
2017-01-18 12:17:15,990 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-5682fe3c) (logid:901efb52) End cleanup
expired async-jobs
2017-01-18 12:17:16,046 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
(RouterStatusMonitor-1:ctx-7ad0b18a) (logid:b5b19f2b) Found 0 routers to
update status.
2017-01-18 12:17:16,047 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
(RouterStatusMonitor-1:ctx-7ad0b18a) (logid:b5b19f2b) Found 0 VPC networks
to update Redundant State.
2017-01-18 12:17:16,049 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
(RouterStatusMonitor-1:ctx-7ad0b18a) (logid:b5b19f2b) Found 0 networks to
update RvR status.
2017-01-18 12:17:16,097 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
(RouterStatusMonitor-1:ctx-10a8aefd) (logid:b09ffa05) Found 0 routers to
update status.
2017-01-18 12:17:16,099 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
(RouterStatusMonitor-1:ctx-10a8aefd) (logid:b09ffa05) Found 0 VPC networks
to update Redundant State.
2017-01-18 12:17:16,101 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
(RouterStatusMonitor-1:ctx-10a8aefd) (logid:b09ffa05) Found 0 networks to
update RvR status.
2017-01-18 12:17:19,799 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentManager-Handler-3:null) (logid:) SeqA 3-438476: Processing Seq
3-438476: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
[{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,"_loadInfo":"{\n
\"connections\": []\n}","wait":0}}] }
2017-01-18 12:17:19,925 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentManager-Handler-3:null) (logid:) SeqA 3-438476: Sending Seq
3-438476: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1, Flags:
100010,
[{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}] }
2017-01-18 12:17:25,985 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-a4c8ceb4) (logid:d0d85ed6) Begin cleanup
expired async-jobs
2017-01-18 12:17:25,991 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-a4c8ceb4) (logid:d0d85ed6) End cleanup
expired async-jobs
2017-01-18 12:17:29,800 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentManager-Handler-7:null) (logid:) SeqA 3-438477: Processing Seq
3-438477: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
[{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,"_loadInfo":"{\n
\"connections\": []\n}","wait":0}}] }
2017-01-18 12:17:29,860 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentManager-Handler-7:null) (logid:) SeqA 3-438477: Sending Seq
3-438477: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1, Flags:
100010,
[{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}] }
2017-01-18 12:17:30,844 DEBUG [o.a.c.s.SecondaryStorageManagerImpl]
(secstorage-1:ctx-f2d92c7a) (logid:6bcbea1f) Zone 1 is ready to launch
secondary storage VM
2017-01-18 12:17:31,095 DEBUG [c.c.c.ConsoleProxyManagerImpl]
(consoleproxy-1:ctx-5c5c1698) (logid:a199e87c) Zone 1 is ready to launch
console proxy
2017-01-18 12:17:34,801 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentManager-Handler-4:null) (logid:) SeqA 3-438478: Processing Seq
3-438478: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
[{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,"_loadInfo":"{\n
\"connections\": []\n}","wait":0}}] }
2017-01-18 12:17:34,853 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentManager-Handler-4:null) (logid:) SeqA 3-438478: Sending Seq
3-438478: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1, Flags:
100010,
[{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}] }
2017-01-18 12:17:35,534 DEBUG [c.c.s.StatsCollector]
(StatsCollector-4:ctx-37da523c) (logid:aecb957a) AutoScaling Monitor is
running...
2017-01-18 12:17:35,984 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-d031d5da) (logid:722c8ae3) Begin cleanup
expired async-jobs
2017-01-18 12:17:35,990 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-d031d5da) (logid:722c8ae3) End cleanup
expired async-jobs
2017-01-18 12:17:44,802 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentManager-Handler-1:null) (logid:) SeqA 3-438479: Processing Seq
3-438479: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
[{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,"_loadInfo":"{\n
\"connections\": []\n}","wait":0}}] }
2017-01-18 12:17:44,863 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentManager-Handler-1:null) (logid:) SeqA 3-438479: Sending Seq
3-438479: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1, Flags:
100010,
[{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}] }
2017-01-18 12:17:45,754 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentManager-Handler-12:null) (logid:) Ping from 1(iuatc)
2017-01-18 12:17:45,754 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
(AgentManager-Handler-12:null) (logid:) Process host VM state report from
ping process. host: 1
2017-01-18 12:17:45,760 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
(AgentManager-Handler-12:null) (logid:) Process VM state report. host: 1,
number of records in report: 2
2017-01-18 12:17:45,760 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
(AgentManager-Handler-12:null) (logid:) VM state report. host: 1, vm id:
1, power state: PowerOn
2017-01-18 12:17:45,763 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
(AgentManager-Handler-12:null) (logid:) VM power state does not change,
skip DB writing. vm id: 1
2017-01-18 12:17:45,763 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
(AgentManager-Handler-12:null) (logid:) VM state report. host: 1, vm id:
2, power state: PowerOn
2017-01-18 12:17:45,766 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
(AgentManager-Handler-12:null) (logid:) VM power state does not change,
skip DB writing. vm id: 2
2017-01-18 12:17:45,769 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
(AgentManager-Handler-12:null) (logid:) Done with process of VM state
report. host: 1
2017-01-18 12:17:45,985 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-b39e7e84) (logid:d0f90660) Begin cleanup
expired async-jobs
2017-01-18 12:17:45,991 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-b39e7e84) (logid:d0f90660) End cleanup
expired async-jobs
2017-01-18 12:17:46,046 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
(RouterStatusMonitor-1:ctx-36fbb5c2) (logid:d346d004) Found 0 routers to
update status.
2017-01-18 12:17:46,047 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
(RouterStatusMonitor-1:ctx-36fbb5c2) (logid:d346d004) Found 0 VPC networks
to update Redundant State.
2017-01-18 12:17:46,049 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
(RouterStatusMonitor-1:ctx-36fbb5c2) (logid:d346d004) Found 0 networks to
update RvR status.
2017-01-18 12:17:46,097 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
(RouterStatusMonitor-1:ctx-16234697) (logid:beb5fec7) Found 0 routers to
update status.
2017-01-18 12:17:46,099 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
(RouterStatusMonitor-1:ctx-16234697) (logid:beb5fec7) Found 0 VPC networks
to update Redundant State.
2017-01-18 12:17:46,100 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
(RouterStatusMonitor-1:ctx-16234697) (logid:beb5fec7) Found 0 networks to
update RvR status.
2017-01-18 12:17:46,201 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentManager-Handler-2:null) (logid:) Ping from 2(s-2-VM)
2017-01-18 12:17:47,521 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentManager-Handler-14:null) (logid:) Ping from 3(v-1-VM)
2017-01-18 12:17:49,842 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentManager-Handler-13:null) (logid:) SeqA 3-438481: Processing Seq
3-438481: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
[{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,"_loadInfo":"{\n
\"connections\": []\n}","wait":0}}] }
2017-01-18 12:17:49,897 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentManager-Handler-13:null) (logid:) SeqA 3-438481: Sending Seq
3-438481: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1, Flags:
100010,
[{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}] }
2017-01-18 12:17:55,985 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-c0db35b5) (logid:c304f172) Begin cleanup
expired async-jobs
2017-01-18 12:17:55,991 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-c0db35b5) (logid:c304f172) End cleanup
expired async-jobs
2017-01-18 12:17:59,250 DEBUG [c.c.s.StatsCollector]
(StatsCollector-6:ctx-eec84206) (logid:b88e0ce8) VmStatsCollector is
running...
2017-01-18 12:17:59,803 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentManager-Handler-11:null) (logid:) SeqA 3-438482: Processing Seq
3-438482: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
[{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,"_loadInfo":"{\n
\"connections\": []\n}","wait":0}}] }
2017-01-18 12:17:59,857 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentManager-Handler-11:null) (logid:) SeqA 3-438482: Sending Seq
3-438482: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1, Flags:
100010,
[{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}] }
2017-01-18 12:18:00,843 DEBUG [o.a.c.s.SecondaryStorageManagerImpl]
(secstorage-1:ctx-f45be089) (logid:934bd9f7) Zone 1 is ready to launch
secondary storage VM
2017-01-18 12:18:01,094 DEBUG [c.c.c.ConsoleProxyManagerImpl]
(consoleproxy-1:ctx-4b11dfe0) (logid:cce52dbb) Zone 1 is ready to launch
console proxy
2017-01-18 12:18:03,345 DEBUG [c.c.s.StatsCollector]
(StatsCollector-2:ctx-c743d4ee) (logid:c1f62e31) HostStatsCollector is
running...
2017-01-18 12:18:03,406 DEBUG [c.c.a.t.Request]
(StatsCollector-2:ctx-c743d4ee) (logid:c1f62e31) Seq
1-6227070909769882805: Received: { Ans: , MgmtId: 279278805450954, via:
1(iuatc), Ver: v1, Flags: 10, { GetHostStatsAnswer } }
2017-01-18 12:18:04,804 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentManager-Handler-9:null) (logid:) SeqA 3-438483: Processing Seq
3-438483: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
[{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,"_loadInfo":"{\n
\"connections\": []\n}","wait":0}}] }
2017-01-18 12:18:04,858 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentManager-Handler-9:null) (logid:) SeqA 3-438483: Sending Seq
3-438483: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1, Flags:
100010,
[{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}] }
2017-01-18 12:18:05,985 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-149380f2) (logid:53d166f9) Begin cleanup
expired async-jobs
2017-01-18 12:18:05,990 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-149380f2) (logid:53d166f9) End cleanup
expired async-jobs
2017-01-18 12:18:14,805 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentManager-Handler-5:null) (logid:) SeqA 3-438484: Processing Seq
3-438484: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
[{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,"_loadInfo":"{\n
\"connections\": []\n}","wait":0}}] }
2017-01-18 12:18:14,951 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentManager-Handler-5:null) (logid:) SeqA 3-438484: Sending Seq
3-438484: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1, Flags:
100010,
[{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}] }
2017-01-18 12:18:15,985 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-7ccabf83) (logid:2b0f34c4) Begin cleanup
expired async-jobs
2017-01-18 12:18:15,993 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-7ccabf83) (logid:2b0f34c4) End cleanup
expired async-jobs
2017-01-18 12:18:16,046 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
(RouterStatusMonitor-1:ctx-fc5408a0) (logid:bbd625c1) Found 0 routers to
update status.
2017-01-18 12:18:16,048 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
(RouterStatusMonitor-1:ctx-fc5408a0) (logid:bbd625c1) Found 0 VPC networks
to update Redundant State.
2017-01-18 12:18:16,049 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
(RouterStatusMonitor-1:ctx-fc5408a0) (logid:bbd625c1) Found 0 networks to
update RvR status.
2017-01-18 12:18:16,097 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
(RouterStatusMonitor-1:ctx-2c697f6b) (logid:cfa955f2) Found 0 routers to
update status.
2017-01-18 12:18:16,099 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
(RouterStatusMonitor-1:ctx-2c697f6b) (logid:cfa955f2) Found 0 VPC networks
to update Redundant State.
2017-01-18 12:18:16,101 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
(RouterStatusMonitor-1:ctx-2c697f6b) (logid:cfa955f2) Found 0 networks to
update RvR status.
2017-01-18 12:18:24,389 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
Timer:ctx-d699e7ed) (logid:e10673e8) Resetting hosts suitable for
reconnect
2017-01-18 12:18:24,392 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
Timer:ctx-d699e7ed) (logid:e10673e8) Completed resetting hosts suitable
for reconnect
2017-01-18 12:18:24,392 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
Timer:ctx-d699e7ed) (logid:e10673e8) Acquiring hosts for clusters already
owned by this management server
2017-01-18 12:18:24,392 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
Timer:ctx-d699e7ed) (logid:e10673e8) Completed acquiring hosts for
clusters already owned by this management server
2017-01-18 12:18:24,392 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
Timer:ctx-d699e7ed) (logid:e10673e8) Acquiring hosts for clusters not
owned by any management server
2017-01-18 12:18:24,393 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
Timer:ctx-d699e7ed) (logid:e10673e8) Completed acquiring hosts for
clusters not owned by any management server
2017-01-18 12:18:24,807 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentManager-Handler-6:null) (logid:) SeqA 3-438485: Processing Seq
3-438485: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
[{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,"_loadInfo":"{\n
\"connections\": []\n}","wait":0}}] }
2017-01-18 12:18:24,870 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentManager-Handler-6:null) (logid:) SeqA 3-438485: Sending Seq
3-438485: { Ans: , MgmtId: 279278805450954, via: 3, Ver: v1, Flags:
100010, [{"com.cloud.agent.api.AgentControlAnswer":{"result":tru
> Can you please share your management_server logs
>
> On Tue, Jan 17, 2017 at 11:47 PM, \uc870\ub300\ud615 <ca...@renet.kr> wrote:
>
>> Hi, all
>>
>> I am facing secondary storage problem.
>>
>> I am using NFS Server to use primary and secondary storage.
>>
>> After rebooting the host,
>>
>> In Dashboard Cloudstack , it says Secondary Storage 0.00 KB / 0.00 KB
>>
>> Primary Storage is okay.
>>
>>
>>
>> After all, system VMs are not up. It was working fine before rebooting.
>>
>>
>>
>> I have tried several times of restart of Management service.
>>
>>
>>
>> Any ideas?
>>
>>
>>
>> Thanks,
>>
>>
>
Re: Cloudstack 4.7 Secondary Storage problem
Posted by sachin patil <sa...@gmail.com>.
Can you please share your management_server logs
On Tue, Jan 17, 2017 at 11:47 PM, 조대형 <ca...@renet.kr> wrote:
> Hi, all
>
> I am facing secondary storage problem.
>
> I am using NFS Server to use primary and secondary storage.
>
> After rebooting the host,
>
> In Dashboard Cloudstack , it says Secondary Storage 0.00 KB / 0.00 KB
>
> Primary Storage is okay.
>
>
>
> After all, system VMs are not up. It was working fine before rebooting.
>
>
>
> I have tried several times of restart of Management service.
>
>
>
> Any ideas?
>
>
>
> Thanks,
>
>