You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@cloudstack.apache.org by adam <ad...@zsah.net> on 2014/08/27 15:47:38 UTC
Unable to add secondary storage
Hi all,
I am trying to run CS 4.4 on CentOS and add a single KVM host. I have primary storage also on the KVM host, and secondary storage (~200GB) on a different (also CentOS) box. The primary storage is around 2TB and is recognised and added correctly (seen in the CS dashboard). The secondary storage seems to be recognised during the initial setup wizard but it always shows as 0/0Kb on the dashboard - and subsequently the test CentOS template does not download (I am assuming because it has nowhere to download to!).
I am able to mount the secondary storage on the management server and I have copied across KVM template as shown in the Prepare the System VM Template part of the installation guide (http://docs.cloudstack.apache.org/projects/cloudstack-installation/en/latest/installation.html#prepare-the-system-vm-template). I am also making sure to perform the steps:
umount /mnt/secondary
rmdir /mnt/secondary
after copying across the template.
I have previously got this working with all the same hardware, except instead of a KVM host on Ubuntu 14.04 it was Citrix XenServer. In that scenario the secondary storage was in the same location and had no trouble being added by the CS management server. I have installed CS and performed the setup from a fresh CentOS installation 2 times now so I'm wondering what I'm missing as I didn't have this trouble previously.
Looking through the log file when I search for the secondary storage IP I see the following line repeated many times:
There is no secondary storage VM for secondary storage host nfs://10.2.3.155/export/secondary
I have Googled this but I'm unable to resolve from the threads I found.
Any help would be much appreciated. Any more information that would be useful please let me know!
Regards,
Adam
RE: Unable to add secondary storage
Posted by Michael Phillips <mp...@hotmail.com>.
I had a similar issue recently...question, is the agent on the SSVM active?
> From: adam@zsah.net
> To: users@cloudstack.apache.org
> Subject: Unable to add secondary storage
> Date: Wed, 27 Aug 2014 13:47:38 +0000
>
> Hi all,
>
> I am trying to run CS 4.4 on CentOS and add a single KVM host. I have primary storage also on the KVM host, and secondary storage (~200GB) on a different (also CentOS) box. The primary storage is around 2TB and is recognised and added correctly (seen in the CS dashboard). The secondary storage seems to be recognised during the initial setup wizard but it always shows as 0/0Kb on the dashboard - and subsequently the test CentOS template does not download (I am assuming because it has nowhere to download to!).
>
> I am able to mount the secondary storage on the management server and I have copied across KVM template as shown in the Prepare the System VM Template part of the installation guide (http://docs.cloudstack.apache.org/projects/cloudstack-installation/en/latest/installation.html#prepare-the-system-vm-template). I am also making sure to perform the steps:
>
> umount /mnt/secondary
> rmdir /mnt/secondary
>
> after copying across the template.
>
> I have previously got this working with all the same hardware, except instead of a KVM host on Ubuntu 14.04 it was Citrix XenServer. In that scenario the secondary storage was in the same location and had no trouble being added by the CS management server. I have installed CS and performed the setup from a fresh CentOS installation 2 times now so I'm wondering what I'm missing as I didn't have this trouble previously.
>
> Looking through the log file when I search for the secondary storage IP I see the following line repeated many times:
> There is no secondary storage VM for secondary storage host nfs://10.2.3.155/export/secondary
> I have Googled this but I'm unable to resolve from the threads I found.
>
> Any help would be much appreciated. Any more information that would be useful please let me know!
>
> Regards,
> Adam
RE: Unable to add secondary storage
Posted by Michael Phillips <mp...@hotmail.com>.
Do you also have a bunch of errors in your log that shows the following: "SSL: Fail to find the generated keystore. Loading fail-safe one to continue."
> From: adam@zsah.net
> To: users@cloudstack.apache.org
> Subject: RE: Unable to add secondary storage
> Date: Wed, 27 Aug 2014 15:14:31 +0000
>
> Hi Michael,
>
> Thanks for the reply.
>
> No, the agent is not active. Both the SSVM and CPVM are running but they have a "-" under the agent state column.
>
> If I SSH into the SSVM (via link local address, the private and public IPs don't work - If i check /etc/network/interfaces I can see that there's no gateway set for the public or private IPs) and run the ssvm-check.sh script then it comes up with
> ----------
> ERROR: NFS is not currently mounted
> Try manually mounting from inside the VM
> NFS server is 169.254.0.19
> ----------
> That is the link local address of that SSVM though - the Secondary Storage server is on 10.2.3.155
>
> Regards,
> Adam
>
RE: Unable to add secondary storage
Posted by adam <ad...@zsah.net>.
I've just re-installed the hypervisor host but this time with CentOS 6.5 instead of Ubuntu 14.04. I've also re-installed the CS mgmt server from fresh for good measure! But unfortunately I'm getting the same problem where the agents on the SSVM and Console VM do not seem to start! If I log into the SSVM and run the diagnostic command the errors are as below:
ERROR: NFS is not currently mounted
Try manually mounting from inside the VM
NFS server is 169.254.2.70
PING 169.254.2.70 (169.254.2.70): 48 data bytes
56 bytes from 169.254.2.70: icmp_seq=0 ttl=64 time=0.103 ms
56 bytes from 169.254.2.70: icmp_seq=1 ttl=64 time=0.076 ms
--- 169.254.2.70 ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max/stddev = 0.076/0.089/0.103/0.000 ms
Good: Can ping nfs server
================================================
Management server is 10.25.10.231. Checking connectivity.
ERROR: Cannot connect to 10.25.10.231 port 8250
It is correct in that I cannot seem to ping the CS mgmt server from this VM - but I can ping it from the hypervisor host and they are on the same network! However to get into the SSVM I had to use the link local address from the hypervisor host.
RE: Unable to add secondary storage
Posted by adam <ad...@zsah.net>.
Hi Michael,
Thanks for your response and suggested action. No, this is not production! Just evaluating at the moment.
I ran the command on the management server with the arguments swapped out where necessary then restarted. I then killed both system VMs, and CS recreated them. However still the agent state is listed as "-". Are there further steps I need to take or is there another possible solution to my problem? I can't find anything in the logs which looks like it might be helpful in finding out why - but then I'm also not sure what to search for. I tried searching by VM names and the only WARN I could see came shortly after the console VM allocation DEBUG statements saying:
Was unable to find lock for the key vm_instance3 and thread id xxxxx
Regards,
Adam
RE: Unable to add secondary storage
Posted by Michael Phillips <mp...@hotmail.com>.
I did the following to fix my issue which presented the same way...
Run the following command "keytool -genkey -keystore /etc/cloudstack/management/cloudmanagementserver.keystore -storepass vmops.com -keypass vmops.com -keyalg RSA -validity 3650 -dname cn="CloudstackUser",ou="yourserver FQDN",o="yourserver FQDN",c="Unknown" then restart cloudstack. You probably will also need to destroy the SSVM and console proxy VM, and let cloudstack recreate them.
This environment is not in production right?
> From: adam@zsah.net
> To: users@cloudstack.apache.org
> Subject: RE: Unable to add secondary storage
> Date: Wed, 27 Aug 2014 15:28:24 +0000
>
> Hi Michael,
>
> Yes! There are many lines in the management-server.log that say "SSL: Fail to find the generated keystore. Loading fail-safe one to continue.". Do you know how I should proceed from here?
>
> Regards,
> Adam
>
>
> -----Original Message-----
> From: Michael Phillips [mailto:mphilli7823@hotmail.com]
> Sent: 27 August 2014 16:19
> To: users@cloudstack.apache.org
> Subject: RE: Unable to add secondary storage
>
>
>
> Do you also have a bunch of errors in your log that shows the following: "SSL: Fail to find the generated keystore. Loading fail-safe one to continue."
>
>
>
> > From: adam@zsah.net<ma...@zsah.net>
>
> > To: users@cloudstack.apache.org<ma...@cloudstack.apache.org>
>
> > Subject: RE: Unable to add secondary storage
>
> > Date: Wed, 27 Aug 2014 15:14:31 +0000
>
> >
>
> > Hi Michael,
>
> >
>
> > Thanks for the reply.
>
> >
>
> > No, the agent is not active. Both the SSVM and CPVM are running but they have a "-" under the agent state column.
>
> >
>
> > If I SSH into the SSVM (via link local address, the private and public
>
> > IPs don't work - If i check /etc/network/interfaces I can see that
>
> > there's no gateway set for the public or private IPs) and run the
>
> > ssvm-check.sh script then it comes up with
>
> > ----------
>
> > ERROR: NFS is not currently mounted
>
> > Try manually mounting from inside the VM NFS server is 169.254.0.19
>
> > ----------
>
> > That is the link local address of that SSVM though - the Secondary
>
> > Storage server is on 10.2.3.155
>
> >
>
> > Regards,
>
> > Adam
>
> >
>
>
RE: Unable to add secondary storage
Posted by adam <ad...@zsah.net>.
Hi Michael,
Yes! There are many lines in the management-server.log that say "SSL: Fail to find the generated keystore. Loading fail-safe one to continue.". Do you know how I should proceed from here?
Regards,
Adam
-----Original Message-----
From: Michael Phillips [mailto:mphilli7823@hotmail.com]
Sent: 27 August 2014 16:19
To: users@cloudstack.apache.org
Subject: RE: Unable to add secondary storage
Do you also have a bunch of errors in your log that shows the following: "SSL: Fail to find the generated keystore. Loading fail-safe one to continue."
> From: adam@zsah.net<ma...@zsah.net>
> To: users@cloudstack.apache.org<ma...@cloudstack.apache.org>
> Subject: RE: Unable to add secondary storage
> Date: Wed, 27 Aug 2014 15:14:31 +0000
>
> Hi Michael,
>
> Thanks for the reply.
>
> No, the agent is not active. Both the SSVM and CPVM are running but they have a "-" under the agent state column.
>
> If I SSH into the SSVM (via link local address, the private and public
> IPs don't work - If i check /etc/network/interfaces I can see that
> there's no gateway set for the public or private IPs) and run the
> ssvm-check.sh script then it comes up with
> ----------
> ERROR: NFS is not currently mounted
> Try manually mounting from inside the VM NFS server is 169.254.0.19
> ----------
> That is the link local address of that SSVM though - the Secondary
> Storage server is on 10.2.3.155
>
> Regards,
> Adam
>
RE: Unable to add secondary storage
Posted by adam <ad...@zsah.net>.
Hi Michael,
Thanks for the reply.
No, the agent is not active. Both the SSVM and CPVM are running but they have a "-" under the agent state column.
If I SSH into the SSVM (via link local address, the private and public IPs don't work - If i check /etc/network/interfaces I can see that there's no gateway set for the public or private IPs) and run the ssvm-check.sh script then it comes up with
----------
ERROR: NFS is not currently mounted
Try manually mounting from inside the VM
NFS server is 169.254.0.19
----------
That is the link local address of that SSVM though - the Secondary Storage server is on 10.2.3.155
Regards,
Adam