You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@cloudstack.apache.org by me...@students.iitmandi.ac.in on 2017/02/07 10:03:39 UTC
Re: Cloudstack 4.7 Secondary Storage problem
Hi,
So I was able to fix the error (Secondary storage shows 0 in dashboard). I
simply logged into the ssvm and ran the health checks, which said NFS
currently not mounted. So I flushed the iptables on the host of my
cloudstack setup( which also happens to be the NFS server and Management
server) and then retried the command "mount" in ssvm and it had mounted.
So I restarted the services and now I am able to view the Secondary
Storage 6.33GB/50GB.
However, When I try too download and ISO or upload. It fails.
When I try to download the default XS-tools.iso it gives me an error
saying "Unable to extract the ISO xs-tools.iso as it resides on host and
not on SSVM"
If I try to upload any iso,it again fails and in the status there is
nothing that is updated.
When I checked the ssvm troubleshooting steps
It read that I need to fix up my secstorage.allowed.internal.sites.
Now the problem is that I have given it the ip of my host(which also acts
as a Management server,NFS server and Mysql Database) : -10.8.19.111/32
While the range of ips I gave to Pods and guest VMs was 10.8.170.x range.
Which means that even ssvm has an IP in that range.
What should I do?
> Merlin,
>
> Can you actually ping 10.8.19.111? Have you set up the NFS shares
> according to CloudStack specification ( /export
> <CIDR>(rw,async,no_root_squash,no_subtree_check) )? What is the contents
> of your /etc/exports file?
>
> In short it looks like you have either a connectivity or permissions issue
> \u2013 which is unrelated to CloudStack. Until you have this sorted you
> won\u2019t be able to spin up system VMs etc. With regards to ACLs on your
> NFS share \u2013 keep in mind you need to open this up to any IP address in
> your management IP range \u2013 since the SSVM can appear on any IP in this.
>
> Regards,
> Dag Sonstebo
> Cloud Architect
> ShapeBlue
>
> On 18/01/2017, 12:10, "merlin@students.iitmandi.ac.in"
> <me...@students.iitmandi.ac.in> wrote:
>
> It did not throw me any error this time, however, there was no
> response
> either. The cursor just kept blinking for a very long time. untill I
> did
> ctrl+c.
>
> Also, a brief observation: when I write mount in my nfs server:
>
> I see three lines at the end
> 10.8.19.111:/export/primary on
> /mnt/38454858-08bb-3e92-88c9-354465ad3715
> type nfs
> (rw,noac,actimeo=0,vers=4,addr=10.8.19.111,clientaddr=10.8.19.111)
> 10.8.19.111:/export/primary on /mnt/primary type nfs
> (rw,rsize=8192,wsize=8192,timeo=14,intr,vers=3,noac,actimeo=0,addr=10.8.19.111)
> 10.8.19.111:/export/secondary on /mnt/secondary type nfs
> (rw,rsize=8192,wsize=8192,timeo=14,intr,vers=3,noac,actimeo=0,addr=10.8.19.111)
>
> Which used to be just 2 when I was using cloudstack version 4.4. I
> recently installed 4.8 and I see this after clean installation.
>
> > try "mount -t nfs 10.8.19.111:/export/secondary/ /tmp/secondary"
> >
> > *Vivek Kumar*
> > Virtualization and Cloud Consultant
> >
> > [image: http://www.indiqus.com/images/logo.jpg]
> <http://www.indiqus.com/>
> > *I*ndi*Q*us Technologies Pvt Ltd
> > A-98, LGF, C.R.Park, New Delhi - 110019
> > *O* +91 11 4055 1411 | *M* +91 7503460090
> > www.indiqus.com <http://www.indiqus.com/>
> >
> > On Wed, Jan 18, 2017 at 5:23 PM, <me...@students.iitmandi.ac.in>
> wrote:
> >
> >> Hi,
> >>
> >> I tried like you said: here is the result
> >>
> >> root@s-2-VM:~# mount -t NFS 10.8.19.111:/export/secondary/
> >> /tmp/secondary/
> >> mount: unknown filesystem type 'NFS'
> >> root@s-2-VM:~# mount -t 10.8.19.111:/export/secondary/
> /tmp/secondary/
> >> nfs
> >> mount: mount point nfs does not exist
> >>
> >> I might be running it wrong. Also, i checked the ssvm for the nfs
> >> package
> >> and neither was it there, nor could i download.
> >> Please help
> >>
> >>
> >> > Hi Merlin,
> >> >
> >> > What Paul means is you need to log on to the SSVM, then do
> something
> >> like
> >> > this:
> >> >
> >> > mkdir /tmp/secondary
> >> > mount â\u20ac\u201ct NFS <NFS server IP>:/<NFS/path/here>/
> /tmp/secondary
> >> >
> >> > This will tell you if your NFS permissions are configured
> correctly.
> >> >
> >> > Regards,
> >> > Dag Sonstebo
> >> > Cloud Architect
> >> > ShapeBlue
> >> >
> >> > On 18/01/2017, 09:29, "merlin@students.iitmandi.ac.in"
> >> > <me...@students.iitmandi.ac.in> wrote:
> >> >
> >> > Also when you say mount from ssvm. Does it mean this?
> >> > Log into ssvm and run "mount":- the reply is down below. I am
> not
> >> able
> >> > to
> >> > understand much even from the documentation,
> >> > I am sorry for the lack of full knowledge in this concern.
> >> >
> >> >
> >> > sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
> >> > proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
> >> > udev on /dev type devtmpfs
> >> > (rw,relatime,size=10240k,nr_inodes=62128,mode=755)
> >> > devpts on /dev/pts type devpts
> >> > (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
> >> > tmpfs on /run type tmpfs
> >> > (rw,nosuid,noexec,relatime,size=50896k,mode=755)
> >> > /dev/disk/by-uuid/30c81d3d-ee9f-4a88-81c1-5f349b22ba1d on /
> type
> >> ext4
> >> > (rw,relatime,errors=remount-ro,user_xattr,barrier=1,data=ordered)
> >> > tmpfs on /run/lock type tmpfs
> >> > (rw,nosuid,nodev,noexec,relatime,size=5120k)
> >> > tmpfs on /run/shm type tmpfs
> >> > (rw,nosuid,nodev,noexec,relatime,size=211960k)
> >> > /dev/vda1 on /boot type ext4
> >> > (rw,relatime,user_xattr,barrier=1,data=ordered)
> >> > /dev/vda6 on /home type ext4
> >> > (rw,relatime,user_xattr,barrier=1,data=ordered)
> >> > /dev/vda8 on /opt type ext4
> >> > (rw,relatime,user_xattr,barrier=1,data=ordered)
> >> > /dev/vda11 on /tmp type ext4
> >> > (rw,relatime,user_xattr,barrier=1,data=ordered)
> >> > /dev/vda7 on /usr type ext4
> >> > (rw,relatime,user_xattr,barrier=1,data=ordered)
> >> > /dev/vda9 on /var type ext4
> >> > (rw,relatime,user_xattr,barrier=1,data=ordered)
> >> > /dev/vda10 on /var/log type ext4
> >> > (rw,relatime,user_xattr,barrier=1,data=ordered)
> >> > rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs
> >> (rw,relatime)
> >> >
> >> > > Hi Merlin,
> >> > >
> >> > > To troubleshoot, try mounting the NFS yourself from the
> SSVM and
> >> get
> >> > that
> >> > > working.
> >> > >
> >> > > the likely problem is permissions on the NFS mounts. Either
> the
> >> IP
> >> > of the
> >> > > SSVM is not allowed or the permissions don't include
> >> no_root_squash
> >> > (or
> >> > > equivalent) the SSVM is probably connecting as 'cloud', so
> >> > connections
> >> > > have to be mapped to 'root'. Finally the SSVM only
> reliably
> >> > supports
> >> > > NFSv3
> >> > >
> >> > >
> >> > >
> >> > >
> >> > > Kind regards,
> >> > >
> >> > > Paul Angus
> >> > >
> >> > > paul.angus@shapeblue.comÃ\u201a
> >> > > www.shapeblue.com
> >> > > 53 Chandos Place, Covent Garden, London WC2N 4HSUK
> >> > > @shapeblue
> >> > >
> >> > >
> >> > >
> >> > >
> >> > > -----Original Message-----
> >> > > From: merlin@students.iitmandi.ac.in
> >> > > [mailto:merlin@students.iitmandi.ac.in]
> >> > > Sent: 18 January 2017 08:48
> >> > > To: users@cloudstack.apache.org
> >> > > Subject: Re: Cloudstack 4.7 Secondary Storage problem
> >> > >
> >> > > And this is the result of check.sh
> >> > >
> >> > > 2 packets transmitted, 2 packets received, 0% packet loss
> >> > round-trip
> >> > > min/avg/max/stddev = 0.400/0.550/0.701/0.151 ms
> >> > > Good: Can ping DNS server
> >> > > ================================================
> >> > > Good: DNS resolves download.cloud.com
> >> > > ================================================
> >> > > ERROR: NFS is not currently mounted
> >> > > Try manually mounting from inside the VM NFS server is
> 10.8.0.1
> >> > PING
> >> > > 10.8.0.1 (10.8.0.1): 48 data bytes
> >> > > 56 bytes from 10.8.0.1: icmp_seq=0 ttl=64 time=0.456 ms
> >> > > 56 bytes from 10.8.0.1: icmp_seq=1 ttl=64 time=0.708 ms
> >> > > --- 10.8.0.1 ping statistics ---
> >> > > 2 packets transmitted, 2 packets received, 0% packet loss
> >> > round-trip
> >> > > min/avg/max/stddev = 0.456/0.582/0.708/0.126 ms
> >> > > Good: Can ping nfs server
> >> > > ================================================
> >> > > Management server is 10.8.19.111. Checking connectivity.
> >> > > Good: Can connect to management server port 8250
> >> > > ================================================
> >> > > Good: Java process is running
> >> > > ================================================
> >> > > Tests Complete. Look for ERROR or WARNING above.
> >> > >
> >> > >
> >> > > The problem is that while installation I did mount nfs?
> >> > > I dont understand what else to do next
> >> > >
> >> > >
> >> > >
> >> > >> Hi, even I am facing the same problem.. and here are the
> logs
> >> from
> >> > >> today:
> >> > >>
> >> > >>
> >> > >>
> >> > >> 2017-01-18 12:16:46,100 DEBUG
> >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> > >> (RouterStatusMonitor-1:ctx-bc5cb4cd) (logid:0ffac658)
> Found 0
> >> > networks
> >> > >> to update RvR status.
> >> > >> 2017-01-18 12:16:46,200 DEBUG [c.c.a.m.AgentManagerImpl]
> >> > >> (AgentManager-Handler-11:null) (logid:) Ping from
> 2(s-2-VM)
> >> > >> 2017-01-18 12:16:47,521 DEBUG [c.c.a.m.AgentManagerImpl]
> >> > >> (AgentManager-Handler-15:null) (logid:) Ping from
> 3(v-1-VM)
> >> > >> 2017-01-18 12:16:49,834 DEBUG [c.c.a.m.AgentManagerImpl]
> >> > >> (AgentManager-Handler-9:null) (logid:) SeqA 3-438472:
> >> Processing
> >> > Seq
> >> > >> 3-438472: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags:
> 11,
> >> > >> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand"
> >> :{"_proxyVmId":1,
> >> > >> "_loadInfo":"{\n
> >> > >> \"connections\": []\n}","wait":0}}] }
> >> > >> 2017-01-18 12:16:49,887 DEBUG [c.c.a.m.AgentManagerImpl]
> >> > >> (AgentManager-Handler-9:null) (logid:) SeqA 3-438472:
> Sending
> >> Seq
> >> > >> 3-438472: { Ans: , MgmtId: 279278805450954, via: 3, Ver:
> v1,
> >> > Flags:
> >> > >> 100010,
> >> > >> [{"com.cloud.agent.api.AgentControlAnswer":{"result":
> >> true,"wait":0}}]
> >> > >> }
> >> > >> 2017-01-18 12:16:54,390 DEBUG [c.c.h.d.HostDaoImpl]
> >> > >> (ClusteredAgentManager
> >> > >> Timer:ctx-8464e4b3) (logid:1f15c721) Resetting hosts
> suitable
> >> for
> >> > >> reconnect
> >> > >> 2017-01-18 12:16:54,392 DEBUG [c.c.h.d.HostDaoImpl]
> >> > >> (ClusteredAgentManager
> >> > >> Timer:ctx-8464e4b3) (logid:1f15c721) Completed resetting
> hosts
> >> > >> suitable for reconnect
> >> > >> 2017-01-18 12:16:54,392 DEBUG [c.c.h.d.HostDaoImpl]
> >> > >> (ClusteredAgentManager
> >> > >> Timer:ctx-8464e4b3) (logid:1f15c721) Acquiring hosts for
> >> clusters
> >> > >> already owned by this management server
> >> > >> 2017-01-18 12:16:54,393 DEBUG [c.c.h.d.HostDaoImpl]
> >> > >> (ClusteredAgentManager
> >> > >> Timer:ctx-8464e4b3) (logid:1f15c721) Completed acquiring
> hosts
> >> for
> >> > >> clusters already owned by this management server
> >> > >> 2017-01-18 12:16:54,393 DEBUG [c.c.h.d.HostDaoImpl]
> >> > >> (ClusteredAgentManager
> >> > >> Timer:ctx-8464e4b3) (logid:1f15c721) Acquiring hosts for
> >> clusters
> >> > not
> >> > >> owned by any management server
> >> > >> 2017-01-18 12:16:54,394 DEBUG [c.c.h.d.HostDaoImpl]
> >> > >> (ClusteredAgentManager
> >> > >> Timer:ctx-8464e4b3) (logid:1f15c721) Completed acquiring
> hosts
> >> for
> >> > >> clusters not owned by any management server
> >> > >> 2017-01-18 12:16:54,795 DEBUG [c.c.a.m.AgentManagerImpl]
> >> > >> (AgentManager-Handler-5:null) (logid:) SeqA 3-438473:
> >> Processing
> >> > Seq
> >> > >> 3-438473: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags:
> 11,
> >> > >> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand"
> >> :{"_proxyVmId":1,
> >> > >> "_loadInfo":"{\n
> >> > >> \"connections\": []\n}","wait":0}}] }
> >> > >> 2017-01-18 12:16:54,854 DEBUG [c.c.a.m.AgentManagerImpl]
> >> > >> (AgentManager-Handler-5:null) (logid:) SeqA 3-438473:
> Sending
> >> Seq
> >> > >> 3-438473: { Ans: , MgmtId: 279278805450954, via: 3, Ver:
> v1,
> >> > Flags:
> >> > >> 100010,
> >> > >> [{"com.cloud.agent.api.AgentControlAnswer":{"result":
> >> true,"wait":0}}]
> >> > >> }
> >> > >> 2017-01-18 12:16:55,984 INFO
> [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> > >> (AsyncJobMgr-Heartbeat-1:ctx-d0469453) (logid:af5d4f47)
> Begin
> >> > cleanup
> >> > >> expired async-jobs
> >> > >> 2017-01-18 12:16:55,991 INFO
> [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> > >> (AsyncJobMgr-Heartbeat-1:ctx-d0469453) (logid:af5d4f47)
> End
> >> > cleanup
> >> > >> expired async-jobs
> >> > >> 2017-01-18 12:16:59,245 DEBUG [c.c.s.StatsCollector]
> >> > >> (StatsCollector-1:ctx-68069dff) (logid:a7a6974d)
> >> VmStatsCollector
> >> > is
> >> > >> running...
> >> > >> 2017-01-18 12:17:00,841 DEBUG
> >> > [o.a.c.s.SecondaryStorageManagerImpl]
> >> > >> (secstorage-1:ctx-4e101aa5) (logid:65f1f0cb) Zone 1 is
> ready to
> >> > launch
> >> > >> secondary storage VM
> >> > >> 2017-01-18 12:17:01,095 DEBUG
> [c.c.c.ConsoleProxyManagerImpl]
> >> > >> (consoleproxy-1:ctx-393828fa) (logid:7b3d0491) Zone 1 is
> ready
> >> to
> >> > >> launch console proxy
> >> > >> 2017-01-18 12:17:03,283 DEBUG [c.c.s.StatsCollector]
> >> > >> (StatsCollector-3:ctx-cc1f58a1) (logid:3ae41340)
> >> HostStatsCollector
> >> > is
> >> > >> running...
> >> > >> 2017-01-18 12:17:03,343 DEBUG [c.c.a.t.Request]
> >> > >> (StatsCollector-3:ctx-cc1f58a1) (logid:3ae41340) Seq
> >> > >> 1-6227070909769882804: Received: { Ans: , MgmtId:
> >> > 279278805450954,
> >> > >> via:
> >> > >> 1(iuatc), Ver: v1, Flags: 10, { GetHostStatsAnswer } }
> >> > >> 2017-01-18 12:17:04,796 DEBUG [c.c.a.m.AgentManagerImpl]
> >> > >> (AgentManager-Handler-10:null) (logid:) SeqA 3-438474:
> >> Processing
> >> > Seq
> >> > >> 3-438474: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags:
> 11,
> >> > >> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand"
> >> :{"_proxyVmId":1,
> >> > >> "_loadInfo":"{\n
> >> > >> \"connections\": []\n}","wait":0}}] }
> >> > >> 2017-01-18 12:17:04,847 DEBUG [c.c.a.m.AgentManagerImpl]
> >> > >> (AgentManager-Handler-10:null) (logid:) SeqA 3-438474:
> Sending
> >> Seq
> >> > >> 3-438474: { Ans: , MgmtId: 279278805450954, via: 3, Ver:
> v1,
> >> > Flags:
> >> > >> 100010,
> >> > >> [{"com.cloud.agent.api.AgentControlAnswer":{"result":
> >> true,"wait":0}}]
> >> > >> }
> >> > >> 2017-01-18 12:17:05,985 INFO
> [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> > >> (AsyncJobMgr-Heartbeat-1:ctx-7f0de3b8) (logid:e9bbd63a)
> Begin
> >> > cleanup
> >> > >> expired async-jobs
> >> > >> 2017-01-18 12:17:05,991 INFO
> [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> > >> (AsyncJobMgr-Heartbeat-1:ctx-7f0de3b8) (logid:e9bbd63a)
> End
> >> > cleanup
> >> > >> expired async-jobs
> >> > >> 2017-01-18 12:17:14,798 DEBUG [c.c.a.m.AgentManagerImpl]
> >> > >> (AgentManager-Handler-8:null) (logid:) SeqA 3-438475:
> >> Processing
> >> > Seq
> >> > >> 3-438475: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags:
> 11,
> >> > >> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand"
> >> :{"_proxyVmId":1,
> >> > >> "_loadInfo":"{\n
> >> > >> \"connections\": []\n}","wait":0}}] }
> >> > >> 2017-01-18 12:17:14,857 DEBUG [c.c.a.m.AgentManagerImpl]
> >> > >> (AgentManager-Handler-8:null) (logid:) SeqA 3-438475:
> Sending
> >> Seq
> >> > >> 3-438475: { Ans: , MgmtId: 279278805450954, via: 3, Ver:
> v1,
> >> > Flags:
> >> > >> 100010,
> >> > >> [{"com.cloud.agent.api.AgentControlAnswer":{"result":
> >> true,"wait":0}}]
> >> > >> }
> >> > >> 2017-01-18 12:17:15,984 INFO
> [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> > >> (AsyncJobMgr-Heartbeat-1:ctx-5682fe3c) (logid:901efb52)
> Begin
> >> > cleanup
> >> > >> expired async-jobs
> >> > >> 2017-01-18 12:17:15,990 INFO
> [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> > >> (AsyncJobMgr-Heartbeat-1:ctx-5682fe3c) (logid:901efb52)
> End
> >> > cleanup
> >> > >> expired async-jobs
> >> > >> 2017-01-18 12:17:16,046 DEBUG
> >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> > >> (RouterStatusMonitor-1:ctx-7ad0b18a) (logid:b5b19f2b)
> Found 0
> >> > routers
> >> > >> to update status.
> >> > >> 2017-01-18 12:17:16,047 DEBUG
> >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> > >> (RouterStatusMonitor-1:ctx-7ad0b18a) (logid:b5b19f2b)
> Found 0
> >> VPC
> >> > >> networks to update Redundant State.
> >> > >> 2017-01-18 12:17:16,049 DEBUG
> >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> > >> (RouterStatusMonitor-1:ctx-7ad0b18a) (logid:b5b19f2b)
> Found 0
> >> > networks
> >> > >> to update RvR status.
> >> > >> 2017-01-18 12:17:16,097 DEBUG
> >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> > >> (RouterStatusMonitor-1:ctx-10a8aefd) (logid:b09ffa05)
> Found 0
> >> > routers
> >> > >> to update status.
> >> > >> 2017-01-18 12:17:16,099 DEBUG
> >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> > >> (RouterStatusMonitor-1:ctx-10a8aefd) (logid:b09ffa05)
> Found 0
> >> VPC
> >> > >> networks to update Redundant State.
> >> > >> 2017-01-18 12:17:16,101 DEBUG
> >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> > >> (RouterStatusMonitor-1:ctx-10a8aefd) (logid:b09ffa05)
> Found 0
> >> > networks
> >> > >> to update RvR status.
> >> > >> 2017-01-18 12:17:19,799 DEBUG [c.c.a.m.AgentManagerImpl]
> >> > >> (AgentManager-Handler-3:null) (logid:) SeqA 3-438476:
> >> Processing
> >> > Seq
> >> > >> 3-438476: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags:
> 11,
> >> > >> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand"
> >> :{"_proxyVmId":1,
> >> > >> "_loadInfo":"{\n
> >> > >> \"connections\": []\n}","wait":0}}] }
> >> > >> 2017-01-18 12:17:19,925 DEBUG [c.c.a.m.AgentManagerImpl]
> >> > >> (AgentManager-Handler-3:null) (logid:) SeqA 3-438476:
> Sending
> >> Seq
> >> > >> 3-438476: { Ans: , MgmtId: 279278805450954, via: 3, Ver:
> v1,
> >> > Flags:
> >> > >> 100010,
> >> > >> [{"com.cloud.agent.api.AgentControlAnswer":{"result":
> >> true,"wait":0}}]
> >> > >> }
> >> > >> 2017-01-18 12:17:25,985 INFO
> [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> > >> (AsyncJobMgr-Heartbeat-1:ctx-a4c8ceb4) (logid:d0d85ed6)
> Begin
> >> > cleanup
> >> > >> expired async-jobs
> >> > >> 2017-01-18 12:17:25,991 INFO
> [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> > >> (AsyncJobMgr-Heartbeat-1:ctx-a4c8ceb4) (logid:d0d85ed6)
> End
> >> > cleanup
> >> > >> expired async-jobs
> >> > >> 2017-01-18 12:17:29,800 DEBUG [c.c.a.m.AgentManagerImpl]
> >> > >> (AgentManager-Handler-7:null) (logid:) SeqA 3-438477:
> >> Processing
> >> > Seq
> >> > >> 3-438477: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags:
> 11,
> >> > >> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand"
> >> :{"_proxyVmId":1,
> >> > >> "_loadInfo":"{\n
> >> > >> \"connections\": []\n}","wait":0}}] }
> >> > >> 2017-01-18 12:17:29,860 DEBUG [c.c.a.m.AgentManagerImpl]
> >> > >> (AgentManager-Handler-7:null) (logid:) SeqA 3-438477:
> Sending
> >> Seq
> >> > >> 3-438477: { Ans: , MgmtId: 279278805450954, via: 3, Ver:
> v1,
> >> > Flags:
> >> > >> 100010,
> >> > >> [{"com.cloud.agent.api.AgentControlAnswer":{"result":
> >> true,"wait":0}}]
> >> > >> }
> >> > >> 2017-01-18 12:17:30,844 DEBUG
> >> > [o.a.c.s.SecondaryStorageManagerImpl]
> >> > >> (secstorage-1:ctx-f2d92c7a) (logid:6bcbea1f) Zone 1 is
> ready to
> >> > launch
> >> > >> secondary storage VM
> >> > >> 2017-01-18 12:17:31,095 DEBUG
> [c.c.c.ConsoleProxyManagerImpl]
> >> > >> (consoleproxy-1:ctx-5c5c1698) (logid:a199e87c) Zone 1 is
> ready
> >> to
> >> > >> launch console proxy
> >> > >> 2017-01-18 12:17:34,801 DEBUG [c.c.a.m.AgentManagerImpl]
> >> > >> (AgentManager-Handler-4:null) (logid:) SeqA 3-438478:
> >> Processing
> >> > Seq
> >> > >> 3-438478: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags:
> 11,
> >> > >> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand"
> >> :{"_proxyVmId":1,
> >> > >> "_loadInfo":"{\n
> >> > >> \"connections\": []\n}","wait":0}}] }
> >> > >> 2017-01-18 12:17:34,853 DEBUG [c.c.a.m.AgentManagerImpl]
> >> > >> (AgentManager-Handler-4:null) (logid:) SeqA 3-438478:
> Sending
> >> Seq
> >> > >> 3-438478: { Ans: , MgmtId: 279278805450954, via: 3, Ver:
> v1,
> >> > Flags:
> >> > >> 100010,
> >> > >> [{"com.cloud.agent.api.AgentControlAnswer":{"result":
> >> true,"wait":0}}]
> >> > >> }
> >> > >> 2017-01-18 12:17:35,534 DEBUG [c.c.s.StatsCollector]
> >> > >> (StatsCollector-4:ctx-37da523c) (logid:aecb957a)
> AutoScaling
> >> > Monitor
> >> > >> is running...
> >> > >> 2017-01-18 12:17:35,984 INFO
> [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> > >> (AsyncJobMgr-Heartbeat-1:ctx-d031d5da) (logid:722c8ae3)
> Begin
> >> > cleanup
> >> > >> expired async-jobs
> >> > >> 2017-01-18 12:17:35,990 INFO
> [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> > >> (AsyncJobMgr-Heartbeat-1:ctx-d031d5da) (logid:722c8ae3)
> End
> >> > cleanup
> >> > >> expired async-jobs
> >> > >> 2017-01-18 12:17:44,802 DEBUG [c.c.a.m.AgentManagerImpl]
> >> > >> (AgentManager-Handler-1:null) (logid:) SeqA 3-438479:
> >> Processing
> >> > Seq
> >> > >> 3-438479: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags:
> 11,
> >> > >> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand"
> >> :{"_proxyVmId":1,
> >> > >> "_loadInfo":"{\n
> >> > >> \"connections\": []\n}","wait":0}}] }
> >> > >> 2017-01-18 12:17:44,863 DEBUG [c.c.a.m.AgentManagerImpl]
> >> > >> (AgentManager-Handler-1:null) (logid:) SeqA 3-438479:
> Sending
> >> Seq
> >> > >> 3-438479: { Ans: , MgmtId: 279278805450954, via: 3, Ver:
> v1,
> >> > Flags:
> >> > >> 100010,
> >> > >> [{"com.cloud.agent.api.AgentControlAnswer":{"result":
> >> true,"wait":0}}]
> >> > >> }
> >> > >> 2017-01-18 12:17:45,754 DEBUG [c.c.a.m.AgentManagerImpl]
> >> > >> (AgentManager-Handler-12:null) (logid:) Ping from
> 1(iuatc)
> >> > >> 2017-01-18 12:17:45,754 DEBUG
> >> > [c.c.v.VirtualMachinePowerStateSyncImpl]
> >> > >> (AgentManager-Handler-12:null) (logid:) Process host VM
> state
> >> > report
> >> > >> from ping process. host: 1
> >> > >> 2017-01-18 12:17:45,760 DEBUG
> >> > [c.c.v.VirtualMachinePowerStateSyncImpl]
> >> > >> (AgentManager-Handler-12:null) (logid:) Process VM state
> >> report.
> >> > host:
> >> > >> 1, number of records in report: 2
> >> > >> 2017-01-18 12:17:45,760 DEBUG
> >> > [c.c.v.VirtualMachinePowerStateSyncImpl]
> >> > >> (AgentManager-Handler-12:null) (logid:) VM state report.
> host:
> >> 1,
> >> > vm
> >> > >> id:
> >> > >> 1, power state: PowerOn
> >> > >> 2017-01-18 12:17:45,763 DEBUG
> >> > [c.c.v.VirtualMachinePowerStateSyncImpl]
> >> > >> (AgentManager-Handler-12:null) (logid:) VM power state
> does not
> >> > >> change, skip DB writing. vm id: 1
> >> > >> 2017-01-18 12:17:45,763 DEBUG
> >> > [c.c.v.VirtualMachinePowerStateSyncImpl]
> >> > >> (AgentManager-Handler-12:null) (logid:) VM state report.
> host:
> >> 1,
> >> > vm
> >> > >> id:
> >> > >> 2, power state: PowerOn
> >> > >> 2017-01-18 12:17:45,766 DEBUG
> >> > [c.c.v.VirtualMachinePowerStateSyncImpl]
> >> > >> (AgentManager-Handler-12:null) (logid:) VM power state
> does not
> >> > >> change, skip DB writing. vm id: 2
> >> > >> 2017-01-18 12:17:45,769 DEBUG
> >> > [c.c.v.VirtualMachinePowerStateSyncImpl]
> >> > >> (AgentManager-Handler-12:null) (logid:) Done with process
> of VM
> >> > state
> >> > >> report. host: 1
> >> > >> 2017-01-18 12:17:45,985 INFO
> [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> > >> (AsyncJobMgr-Heartbeat-1:ctx-b39e7e84) (logid:d0f90660)
> Begin
> >> > cleanup
> >> > >> expired async-jobs
> >> > >> 2017-01-18 12:17:45,991 INFO
> [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> > >> (AsyncJobMgr-Heartbeat-1:ctx-b39e7e84) (logid:d0f90660)
> End
> >> > cleanup
> >> > >> expired async-jobs
> >> > >> 2017-01-18 12:17:46,046 DEBUG
> >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> > >> (RouterStatusMonitor-1:ctx-36fbb5c2) (logid:d346d004)
> Found 0
> >> > routers
> >> > >> to update status.
> >> > >> 2017-01-18 12:17:46,047 DEBUG
> >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> > >> (RouterStatusMonitor-1:ctx-36fbb5c2) (logid:d346d004)
> Found 0
> >> VPC
> >> > >> networks to update Redundant State.
> >> > >> 2017-01-18 12:17:46,049 DEBUG
> >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> > >> (RouterStatusMonitor-1:ctx-36fbb5c2) (logid:d346d004)
> Found 0
> >> > networks
> >> > >> to update RvR status.
> >> > >> 2017-01-18 12:17:46,097 DEBUG
> >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> > >> (RouterStatusMonitor-1:ctx-16234697) (logid:beb5fec7)
> Found 0
> >> > routers
> >> > >> to update status.
> >> > >> 2017-01-18 12:17:46,099 DEBUG
> >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> > >> (RouterStatusMonitor-1:ctx-16234697) (logid:beb5fec7)
> Found 0
> >> VPC
> >> > >> networks to update Redundant State.
> >> > >> 2017-01-18 12:17:46,100 DEBUG
> >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> > >> (RouterStatusMonitor-1:ctx-16234697) (logid:beb5fec7)
> Found 0
> >> > networks
> >> > >> to update RvR status.
> >> > >> 2017-01-18 12:17:46,201 DEBUG [c.c.a.m.AgentManagerImpl]
> >> > >> (AgentManager-Handler-2:null) (logid:) Ping from
> 2(s-2-VM)
> >> > >> 2017-01-18 12:17:47,521 DEBUG [c.c.a.m.AgentManagerImpl]
> >> > >> (AgentManager-Handler-14:null) (logid:) Ping from
> 3(v-1-VM)
> >> > >> 2017-01-18 12:17:49,842 DEBUG [c.c.a.m.AgentManagerImpl]
> >> > >> (AgentManager-Handler-13:null) (logid:) SeqA 3-438481:
> >> Processing
> >> > Seq
> >> > >> 3-438481: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags:
> 11,
> >> > >> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand"
> >> :{"_proxyVmId":1,
> >> > >> "_loadInfo":"{\n
> >> > >> \"connections\": []\n}","wait":0}}] }
> >> > >> 2017-01-18 12:17:49,897 DEBUG [c.c.a.m.AgentManagerImpl]
> >> > >> (AgentManager-Handler-13:null) (logid:) SeqA 3-438481:
> Sending
> >> Seq
> >> > >> 3-438481: { Ans: , MgmtId: 279278805450954, via: 3, Ver:
> v1,
> >> > Flags:
> >> > >> 100010,
> >> > >> [{"com.cloud.agent.api.AgentControlAnswer":{"result":
> >> true,"wait":0}}]
> >> > >> }
> >> > >> 2017-01-18 12:17:55,985 INFO
> [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> > >> (AsyncJobMgr-Heartbeat-1:ctx-c0db35b5) (logid:c304f172)
> Begin
> >> > cleanup
> >> > >> expired async-jobs
> >> > >> 2017-01-18 12:17:55,991 INFO
> [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> > >> (AsyncJobMgr-Heartbeat-1:ctx-c0db35b5) (logid:c304f172)
> End
> >> > cleanup
> >> > >> expired async-jobs
> >> > >> 2017-01-18 12:17:59,250 DEBUG [c.c.s.StatsCollector]
> >> > >> (StatsCollector-6:ctx-eec84206) (logid:b88e0ce8)
> >> VmStatsCollector
> >> > is
> >> > >> running...
> >> > >> 2017-01-18 12:17:59,803 DEBUG [c.c.a.m.AgentManagerImpl]
> >> > >> (AgentManager-Handler-11:null) (logid:) SeqA 3-438482:
> >> Processing
> >> > Seq
> >> > >> 3-438482: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags:
> 11,
> >> > >> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand"
> >> :{"_proxyVmId":1,
> >> > >> "_loadInfo":"{\n
> >> > >> \"connections\": []\n}","wait":0}}] }
> >> > >> 2017-01-18 12:17:59,857 DEBUG [c.c.a.m.AgentManagerImpl]
> >> > >> (AgentManager-Handler-11:null) (logid:) SeqA 3-438482:
> Sending
> >> Seq
> >> > >> 3-438482: { Ans: , MgmtId: 279278805450954, via: 3, Ver:
> v1,
> >> > Flags:
> >> > >> 100010,
> >> > >> [{"com.cloud.agent.api.AgentControlAnswer":{"result":
> >> true,"wait":0}}]
> >> > >> }
> >> > >> 2017-01-18 12:18:00,843 DEBUG
> >> > [o.a.c.s.SecondaryStorageManagerImpl]
> >> > >> (secstorage-1:ctx-f45be089) (logid:934bd9f7) Zone 1 is
> ready to
> >> > launch
> >> > >> secondary storage VM
> >> > >> 2017-01-18 12:18:01,094 DEBUG
> [c.c.c.ConsoleProxyManagerImpl]
> >> > >> (consoleproxy-1:ctx-4b11dfe0) (logid:cce52dbb) Zone 1 is
> ready
> >> to
> >> > >> launch console proxy
> >> > >> 2017-01-18 12:18:03,345 DEBUG [c.c.s.StatsCollector]
> >> > >> (StatsCollector-2:ctx-c743d4ee) (logid:c1f62e31)
> >> HostStatsCollector
> >> > is
> >> > >> running...
> >> > >> 2017-01-18 12:18:03,406 DEBUG [c.c.a.t.Request]
> >> > >> (StatsCollector-2:ctx-c743d4ee) (logid:c1f62e31) Seq
> >> > >> 1-6227070909769882805: Received: { Ans: , MgmtId:
> >> > 279278805450954,
> >> > >> via:
> >> > >> 1(iuatc), Ver: v1, Flags: 10, { GetHostStatsAnswer } }
> >> > >> 2017-01-18 12:18:04,804 DEBUG [c.c.a.m.AgentManagerImpl]
> >> > >> (AgentManager-Handler-9:null) (logid:) SeqA 3-438483:
> >> Processing
> >> > Seq
> >> > >> 3-438483: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags:
> 11,
> >> > >> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand"
> >> :{"_proxyVmId":1,
> >> > >> "_loadInfo":"{\n
> >> > >> \"connections\": []\n}","wait":0}}] }
> >> > >> 2017-01-18 12:18:04,858 DEBUG [c.c.a.m.AgentManagerImpl]
> >> > >> (AgentManager-Handler-9:null) (logid:) SeqA 3-438483:
> Sending
> >> Seq
> >> > >> 3-438483: { Ans: , MgmtId: 279278805450954, via: 3, Ver:
> v1,
> >> > Flags:
> >> > >> 100010,
> >> > >> [{"com.cloud.agent.api.AgentControlAnswer":{"result":
> >> true,"wait":0}}]
> >> > >> }
> >> > >> 2017-01-18 12:18:05,985 INFO
> [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> > >> (AsyncJobMgr-Heartbeat-1:ctx-149380f2) (logid:53d166f9)
> Begin
> >> > cleanup
> >> > >> expired async-jobs
> >> > >> 2017-01-18 12:18:05,990 INFO
> [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> > >> (AsyncJobMgr-Heartbeat-1:ctx-149380f2) (logid:53d166f9)
> End
> >> > cleanup
> >> > >> expired async-jobs
> >> > >> 2017-01-18 12:18:14,805 DEBUG [c.c.a.m.AgentManagerImpl]
> >> > >> (AgentManager-Handler-5:null) (logid:) SeqA 3-438484:
> >> Processing
> >> > Seq
> >> > >> 3-438484: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags:
> 11,
> >> > >> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand"
> >> :{"_proxyVmId":1,
> >> > >> "_loadInfo":"{\n
> >> > >> \"connections\": []\n}","wait":0}}] }
> >> > >> 2017-01-18 12:18:14,951 DEBUG [c.c.a.m.AgentManagerImpl]
> >> > >> (AgentManager-Handler-5:null) (logid:) SeqA 3-438484:
> Sending
> >> Seq
> >> > >> 3-438484: { Ans: , MgmtId: 279278805450954, via: 3, Ver:
> v1,
> >> > Flags:
> >> > >> 100010,
> >> > >> [{"com.cloud.agent.api.AgentControlAnswer":{"result":
> >> true,"wait":0}}]
> >> > >> }
> >> > >> 2017-01-18 12:18:15,985 INFO
> [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> > >> (AsyncJobMgr-Heartbeat-1:ctx-7ccabf83) (logid:2b0f34c4)
> Begin
> >> > cleanup
> >> > >> expired async-jobs
> >> > >> 2017-01-18 12:18:15,993 INFO
> [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> > >> (AsyncJobMgr-Heartbeat-1:ctx-7ccabf83) (logid:2b0f34c4)
> End
> >> > cleanup
> >> > >> expired async-jobs
> >> > >> 2017-01-18 12:18:16,046 DEBUG
> >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> > >> (RouterStatusMonitor-1:ctx-fc5408a0) (logid:bbd625c1)
> Found 0
> >> > routers
> >> > >> to update status.
> >> > >> 2017-01-18 12:18:16,048 DEBUG
> >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> > >> (RouterStatusMonitor-1:ctx-fc5408a0) (logid:bbd625c1)
> Found 0
> >> VPC
> >> > >> networks to update Redundant State.
> >> > >> 2017-01-18 12:18:16,049 DEBUG
> >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> > >> (RouterStatusMonitor-1:ctx-fc5408a0) (logid:bbd625c1)
> Found 0
> >> > networks
> >> > >> to update RvR status.
> >> > >> 2017-01-18 12:18:16,097 DEBUG
> >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> > >> (RouterStatusMonitor-1:ctx-2c697f6b) (logid:cfa955f2)
> Found 0
> >> > routers
> >> > >> to update status.
> >> > >> 2017-01-18 12:18:16,099 DEBUG
> >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> > >> (RouterStatusMonitor-1:ctx-2c697f6b) (logid:cfa955f2)
> Found 0
> >> VPC
> >> > >> networks to update Redundant State.
> >> > >> 2017-01-18 12:18:16,101 DEBUG
> >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> > >> (RouterStatusMonitor-1:ctx-2c697f6b) (logid:cfa955f2)
> Found 0
> >> > networks
> >> > >> to update RvR status.
> >> > >> 2017-01-18 12:18:24,389 DEBUG [c.c.h.d.HostDaoImpl]
> >> > >> (ClusteredAgentManager
> >> > >> Timer:ctx-d699e7ed) (logid:e10673e8) Resetting hosts
> suitable
> >> for
> >> > >> reconnect
> >> > >> 2017-01-18 12:18:24,392 DEBUG [c.c.h.d.HostDaoImpl]
> >> > >> (ClusteredAgentManager
> >> > >> Timer:ctx-d699e7ed) (logid:e10673e8) Completed resetting
> hosts
> >> > >> suitable for reconnect
> >> > >> 2017-01-18 12:18:24,392 DEBUG [c.c.h.d.HostDaoImpl]
> >> > >> (ClusteredAgentManager
> >> > >> Timer:ctx-d699e7ed) (logid:e10673e8) Acquiring hosts for
> >> clusters
> >> > >> already owned by this management server
> >> > >> 2017-01-18 12:18:24,392 DEBUG [c.c.h.d.HostDaoImpl]
> >> > >> (ClusteredAgentManager
> >> > >> Timer:ctx-d699e7ed) (logid:e10673e8) Completed acquiring
> hosts
> >> for
> >> > >> clusters already owned by this management server
> >> > >> 2017-01-18 12:18:24,392 DEBUG [c.c.h.d.HostDaoImpl]
> >> > >> (ClusteredAgentManager
> >> > >> Timer:ctx-d699e7ed) (logid:e10673e8) Acquiring hosts for
> >> clusters
> >> > not
> >> > >> owned by any management server
> >> > >> 2017-01-18 12:18:24,393 DEBUG [c.c.h.d.HostDaoImpl]
> >> > >> (ClusteredAgentManager
> >> > >> Timer:ctx-d699e7ed) (logid:e10673e8) Completed acquiring
> hosts
> >> for
> >> > >> clusters not owned by any management server
> >> > >> 2017-01-18 12:18:24,807 DEBUG [c.c.a.m.AgentManagerImpl]
> >> > >> (AgentManager-Handler-6:null) (logid:) SeqA 3-438485:
> >> Processing
> >> > Seq
> >> > >> 3-438485: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags:
> 11,
> >> > >> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand"
> >> :{"_proxyVmId":1,
> >> > >> "_loadInfo":"{\n
> >> > >> \"connections\": []\n}","wait":0}}] }
> >> > >> 2017-01-18 12:18:24,870 DEBUG [c.c.a.m.AgentManagerImpl]
> >> > >> (AgentManager-Handler-6:null) (logid:) SeqA 3-438485:
> Sending
> >> Seq
> >> > >> 3-438485: { Ans: , MgmtId: 279278805450954, via: 3, Ver:
> v1,
> >> > Flags:
> >> > >> 100010,
> >> [{"com.cloud.agent.api.AgentControlAnswer":{"result":tru
> >> > >>
> >> > >>> Can you please share your management_server logs
> >> > >>>
> >> > >>> On Tue, Jan 17, 2017 at 11:47 PM,
> >> > Ã\u0192¬Ã\u201a¡Ã\u201a°Ã\u0192«Ã\u2026â\u20ac\u2122ââ\u20ac\u0161¬Ã\u0192ÂÃ\u2039Å\u201cââ\u201a¬Â¢
> >> > >>> <ca...@renet.kr> wrote:
> >> > >>>
> >> > >>>> Hi, all
> >> > >>>>
> >> > >>>> I am facing secondary storage problem.
> >> > >>>>
> >> > >>>> I am using NFS Server to use primary and secondary
> storage.
> >> > >>>>
> >> > >>>> After rebooting the host,
> >> > >>>>
> >> > >>>> In Dashboard Cloudstack , it says Secondary Storage 0.00
> KB /
> >> > 0.00
> >> > >>>> KB
> >> > >>>>
> >> > >>>> Primary Storage is okay.
> >> > >>>>
> >> > >>>>
> >> > >>>>
> >> > >>>> After all, system VMs are not up. It was working fine
> before
> >> > >>>> rebooting.
> >> > >>>>
> >> > >>>>
> >> > >>>>
> >> > >>>> I have tried several times of restart of Management
> service.
> >> > >>>>
> >> > >>>>
> >> > >>>>
> >> > >>>> Any ideas?
> >> > >>>>
> >> > >>>>
> >> > >>>>
> >> > >>>> Thanks,
> >> > >>>>
> >> > >>>>
> >> > >>>
> >> > >>
> >> > >>
> >> > >>
> >> > >
> >> > >
> >> > >
> >> >
> >> >
> >> >
> >> >
> >> >
> >> > Dag.Sonstebo@shapeblue.com
> >> > www.shapeblue.com
> >> > 53 Chandos Place, Covent Garden, London WC2N 4HSUK
> >> > @shapeblue
> >> >
> >> >
> >> >
> >> >
> >>
> >>
> >>
> >
>
>
>
>
>
> Dag.Sonstebo@shapeblue.com
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London WC2N 4HSUK
> @shapeblue
>
>
>
>