You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cloudstack.apache.org by "Tutkowski, Mike" <Mi...@netapp.com> on 2016/08/17 23:03:47 UTC
Issue with SSVM on KVM on master
Hi,
I am trying to build a cloud making use of KVM system VMs.
My SSVM VM comes up, but its agent doesn't seem to:
http://imgur.com/a/Fo2ax
Thoughts on what might be going on here?
Thanks!
Mike
Re: Issue with SSVM on KVM on master
Posted by Rajani Karuturi <ra...@apache.org>.
Any errors in the SSVM logs?
~ Rajani
http://cloudplatform.accelerite.com/
On August 18, 2016 at 4:34 AM, Tutkowski, Mike
(Mike.Tutkowski@netapp.com) wrote:
I should have pointed out that this is a Basic Zone. Also, there
is only one KVM host in my one cluster.
________________________________________
From: Tutkowski, Mike <Mi...@netapp.com>
Sent: Wednesday, August 17, 2016 5:03 PM
To: dev@cloudstack.apache.org
Subject: Issue with SSVM on KVM on master
Hi,
I am trying to build a cloud making use of KVM system VMs.
My SSVM VM comes up, but its agent doesn't seem to:
http://imgur.com/a/Fo2ax
Thoughts on what might be going on here?
Thanks!
Mike
Re: Issue with SSVM on KVM on master
Posted by "Tutkowski, Mike" <Mi...@netapp.com>.
Thanks, Frank!
When I went back and made sure to recompile both the management server and my KVM agents without using –Dnoredist, it worked. Perhaps I compiled the management server with –Dnoredist last time.
On 10/11/16, 9:33 AM, "Frank Maximus" <fr...@nuagenetworks.net> wrote:
Make sure that the VMware plug-ins are not in the war on the management
server, when using kvm. If they are, other options get send to the ssvm,
and then it will fail. All has to do with a property premium=true being set
when spring loads the VMware plug-in.
Regards, Frank
On Tue, 11 Oct 2016 16:14 Tutkowski, Mike, <Mi...@netapp.com>
wrote:
As a little correction to what I said, I meant to ask if there is a way in
4.10 to run system VMs without using noredist for the build as I only have
KVM hosts in this particular environment.
This does seem to work fine in 4.5, so it appears something has changed
since that release in this regard.
> On Oct 11, 2016, at 12:15 AM, Tutkowski, Mike <Mi...@netapp.com>
wrote:
>
> Hi Rajani,
>
> I actually ended up getting diverted to other work and am just coming
back to this now. ☺
>
> Do you know if there’s a way in 4.10 to run the KVM agent without needing
the noredist option? In this environment, I only have KVM hosts.
>
> I recently tried this in 4.5 and it worked fine without having to do
anything special. It seems something has changed since 4.5 here.
>
> Thanks!
> Mike
>
> On 8/22/16, 12:43 AM, "Tutkowski, Mike" <Mi...@netapp.com> wrote:
>
> I can try that. Thanks, Rajani!
>
>> On Aug 22, 2016, at 12:17 AM, Rajani Karuturi <ra...@apache.org> wrote:
>>
>> Did you build and run it with noredist option?
>>
>> I see this CNF error when I forget to run jetty with noredist
>> option.
>>
>> ~ Rajani
>> http://cloudplatform.accelerite.com/
>>
>> On August 18, 2016 at 7:57 PM, Tutkowski, Mike
>> (Mike.Tutkowski@netapp.com) wrote:
>> Based on the error message of not being able to find the
>> PremiumSecondaryStorageResource class, this seems to be the
>> following error (#9 on this
>> page:
https://cwiki.apache.org/confluence/display/CLOUDSTACK/SSVM,+templates,+Secondary+storage+troubleshooting
):
>>
>> SSVM agent not running - You see error something like below. Most
>> probably your systemvm didnt get patched with the agent specific
>> code. This patching happens through the ISO - something
>> like systemvm***.iso. Check the size and location of the iso and
>> whether you are not using the old version iso. For XS its on the
>> host so grep for it and for vmware its on secondary storage. Do
>> also check that if its vmware that you built your ssvm using
>> noredist flag for mvn command. For more information check
>> SystemVm.iso
>> 2013-12-20 13:35:12,954 DEBUG [cloud.utils.ProcessUtil]
>> (main:null) Execution is successful.
>> 2013-12-20 13:35:12,960 ERROR [cloud.agent.AgentShell]
>> (main:null) Unable to start agent: Resource class not found:
>> com.cloud.storage.resource.PremiumSecondaryStorageResource due
>> to: java.lang.ClassNotFoundException:
>> com.cloud.storage.resource.PremiumSecondaryStorageResource
>>
>> I do see the ISO, though, on secondary storage here:
>>
>> cloudstack@cloudstack-virtual-machine:/export/secondary/systemvm$
>> ls -l
>> total 83420
>> -rw-rw-r-- 1 cloudstack cloudstack 85420032 Aug 15 16:10
>> systemvm-4.10.0-SNAPSHOT.iso
>> ________________________________________
>> From: Tutkowski, Mike <Mi...@netapp.com>
>> Sent: Thursday, August 18, 2016 8:17 AM
>> To: dev@cloudstack.apache.org
>> Subject: Re: Issue with SSVM on KVM on master
>>
>> In /var/log/cloud/cloud.out, I see the following repeated over
>> and over again (below).
>>
>> I'm running version 4.10.0 of the management server (but the
>> implementation version of the agent looks to be 4.9.0). I do have
>> systemvm-4.10.0-SNAPSHOT.iso accessible at
>> /export/secondary/systemvm (/export/secondary is my one and only
>> secondary storage at the time being).
>>
>> 2016-08-18 14:03:43,633 INFO [cloud.agent.AgentShell]
>> (main:null) Agent started
>> 2016-08-18 14:03:43,637 INFO [cloud.agent.AgentShell]
>> (main:null) Implementation Version is 4.9.0
>> 2016-08-18 14:03:43,637 INFO [cloud.agent.AgentShell]
>> (main:null) agent.properties found at
>> /usr/local/cloud/systemvm/conf/agent.properties
>> 2016-08-18 14:03:43,643 DEBUG [cloud.agent.AgentShell]
>> (main:null) Found property: instance
>> 2016-08-18 14:03:43,643 DEBUG [cloud.agent.AgentShell]
>> (main:null) Found property: resource
>> 2016-08-18 14:03:43,643 INFO [cloud.agent.AgentShell]
>> (main:null) Defaulting to using properties file for storage
>> 2016-08-18 14:03:43,645 INFO [cloud.agent.AgentShell]
>> (main:null) Defaulting to the constant time backoff algorithm
>> 2016-08-18 14:03:43,666 INFO [cloud.utils.LogUtils] (main:null)
>> log4j configuration found at
>> /usr/local/cloud/systemvm/conf/log4j-cloud.xml
>> 2016-08-18 14:03:43,686 INFO [cloud.agent.AgentShell]
>> (main:null) Using default Java settings for IPv6 preference for
>> agent connection
>> 2016-08-18 14:03:43,687 DEBUG [cloud.agent.AgentShell]
>> (main:null) Checking to see if agent.SecStorage.pid exists.
>> 2016-08-18 14:03:43,694 DEBUG [cloud.utils.ProcessUtil]
>> (main:null) environment.properties could not be opened
>> 2016-08-18 14:03:43,703 DEBUG [cloud.utils.ProcessUtil]
>> (main:null) Executing: bash -c echo $PPID
>> 2016-08-18 14:03:43,711 DEBUG [cloud.utils.ProcessUtil]
>> (main:null) Execution is successful.
>> 2016-08-18 14:03:43,726 ERROR [cloud.agent.AgentShell]
>> (main:null) Unable to start agent: Resource class not found:
>> com.cloud.storage.resource.PremiumSecondaryStorageResource due
>> to: java.lang.ClassNotFoundException:
>> com.cloud.storage.resource.PremiumSecondaryStorageResource
>>
>> ________________________________________
>> From: Tutkowski, Mike <Mi...@netapp.com>
>> Sent: Thursday, August 18, 2016 7:54 AM
>> To: dev@cloudstack.apache.org
>> Subject: Re: Issue with SSVM on KVM on master
>>
>> Thanks for the comments, Simon.
>>
>> I went ahead and tried to telnet into 8250 on my management
>> server (first from the SSVM and then from another machine). In
>> both cases, this was the result:
>>
>> telnet 192.168.129.50 8250
>> Trying 192.168.129.50...
>> Connected to 192.168.129.50.
>> Escape character is '^]'.
>> Connection closed by foreign host.
>>
>> It looks like a connection was established, but then shortly
>> after closed. Is this what you were expecting might happen?
>> ________________________________________
>> From: Simon Weller <sw...@ena.com>
>> Sent: Thursday, August 18, 2016 6:39 AM
>> To: dev@cloudstack.apache.org
>> Subject: Re: Issue with SSVM on KVM on master
>>
>> Mike,
>>
>> I wonder whether this could be related to the NIO connection
>> improvements. We saw this type of problem quite a bit when we
>> were testing it (with all types of agents). Typically you'll see
>> the agent logs indicating they can't connect to the management
>> server if this is the case. Try telnetting to 8250 on your mgmt
>> server and see if suddenly your agent starts to work.
>>
>> - Si
>>
>> ________________________________
>> From: Tutkowski, Mike <Mi...@netapp.com>
>> Sent: Wednesday, August 17, 2016 6:04 PM
>> To: dev@cloudstack.apache.org
>> Subject: Re: Issue with SSVM on KVM on master
>>
>> I should have pointed out that this is a Basic Zone. Also, there
>> is only one KVM host in my one cluster.
>> ________________________________________
>> From: Tutkowski, Mike <Mi...@netapp.com>
>> Sent: Wednesday, August 17, 2016 5:03 PM
>> To: dev@cloudstack.apache.org
>> Subject: Issue with SSVM on KVM on master
>>
>> Hi,
>>
>> I am trying to build a cloud making use of KVM system VMs.
>>
>> My SSVM VM comes up, but its agent doesn't seem to:
>>
>> http://imgur.com/a/Fo2ax
>> [http://i.imgur.com/xdP9alM.jpg?fb]<http://imgur.com/a/Fo2ax
>> ( http://i.imgur.com/xdP9alM.jpg?fb%5D<http://imgur.com/a/Fo2ax )>
>>
>> Imgur: The most awesome images on the
>> Internet<http://imgur.com/a/Fo2ax>
>> imgur.com
>> Imgur: The most awesome images on the Internet.
>>
>> Thoughts on what might be going on here?
>>
>> Thanks!
>>
>> Mike
>
>
Re: Issue with SSVM on KVM on master
Posted by Frank Maximus <fr...@nuagenetworks.net>.
Make sure that the VMware plug-ins are not in the war on the management
server, when using kvm. If they are, other options get send to the ssvm,
and then it will fail. All has to do with a property premium=true being set
when spring loads the VMware plug-in.
Regards, Frank
On Tue, 11 Oct 2016 16:14 Tutkowski, Mike, <Mi...@netapp.com>
wrote:
As a little correction to what I said, I meant to ask if there is a way in
4.10 to run system VMs without using noredist for the build as I only have
KVM hosts in this particular environment.
This does seem to work fine in 4.5, so it appears something has changed
since that release in this regard.
> On Oct 11, 2016, at 12:15 AM, Tutkowski, Mike <Mi...@netapp.com>
wrote:
>
> Hi Rajani,
>
> I actually ended up getting diverted to other work and am just coming
back to this now. ☺
>
> Do you know if there’s a way in 4.10 to run the KVM agent without needing
the noredist option? In this environment, I only have KVM hosts.
>
> I recently tried this in 4.5 and it worked fine without having to do
anything special. It seems something has changed since 4.5 here.
>
> Thanks!
> Mike
>
> On 8/22/16, 12:43 AM, "Tutkowski, Mike" <Mi...@netapp.com> wrote:
>
> I can try that. Thanks, Rajani!
>
>> On Aug 22, 2016, at 12:17 AM, Rajani Karuturi <ra...@apache.org> wrote:
>>
>> Did you build and run it with noredist option?
>>
>> I see this CNF error when I forget to run jetty with noredist
>> option.
>>
>> ~ Rajani
>> http://cloudplatform.accelerite.com/
>>
>> On August 18, 2016 at 7:57 PM, Tutkowski, Mike
>> (Mike.Tutkowski@netapp.com) wrote:
>> Based on the error message of not being able to find the
>> PremiumSecondaryStorageResource class, this seems to be the
>> following error (#9 on this
>> page:
https://cwiki.apache.org/confluence/display/CLOUDSTACK/SSVM,+templates,+Secondary+storage+troubleshooting
):
>>
>> SSVM agent not running - You see error something like below. Most
>> probably your systemvm didnt get patched with the agent specific
>> code. This patching happens through the ISO - something
>> like systemvm***.iso. Check the size and location of the iso and
>> whether you are not using the old version iso. For XS its on the
>> host so grep for it and for vmware its on secondary storage. Do
>> also check that if its vmware that you built your ssvm using
>> noredist flag for mvn command. For more information check
>> SystemVm.iso
>> 2013-12-20 13:35:12,954 DEBUG [cloud.utils.ProcessUtil]
>> (main:null) Execution is successful.
>> 2013-12-20 13:35:12,960 ERROR [cloud.agent.AgentShell]
>> (main:null) Unable to start agent: Resource class not found:
>> com.cloud.storage.resource.PremiumSecondaryStorageResource due
>> to: java.lang.ClassNotFoundException:
>> com.cloud.storage.resource.PremiumSecondaryStorageResource
>>
>> I do see the ISO, though, on secondary storage here:
>>
>> cloudstack@cloudstack-virtual-machine:/export/secondary/systemvm$
>> ls -l
>> total 83420
>> -rw-rw-r-- 1 cloudstack cloudstack 85420032 Aug 15 16:10
>> systemvm-4.10.0-SNAPSHOT.iso
>> ________________________________________
>> From: Tutkowski, Mike <Mi...@netapp.com>
>> Sent: Thursday, August 18, 2016 8:17 AM
>> To: dev@cloudstack.apache.org
>> Subject: Re: Issue with SSVM on KVM on master
>>
>> In /var/log/cloud/cloud.out, I see the following repeated over
>> and over again (below).
>>
>> I'm running version 4.10.0 of the management server (but the
>> implementation version of the agent looks to be 4.9.0). I do have
>> systemvm-4.10.0-SNAPSHOT.iso accessible at
>> /export/secondary/systemvm (/export/secondary is my one and only
>> secondary storage at the time being).
>>
>> 2016-08-18 14:03:43,633 INFO [cloud.agent.AgentShell]
>> (main:null) Agent started
>> 2016-08-18 14:03:43,637 INFO [cloud.agent.AgentShell]
>> (main:null) Implementation Version is 4.9.0
>> 2016-08-18 14:03:43,637 INFO [cloud.agent.AgentShell]
>> (main:null) agent.properties found at
>> /usr/local/cloud/systemvm/conf/agent.properties
>> 2016-08-18 14:03:43,643 DEBUG [cloud.agent.AgentShell]
>> (main:null) Found property: instance
>> 2016-08-18 14:03:43,643 DEBUG [cloud.agent.AgentShell]
>> (main:null) Found property: resource
>> 2016-08-18 14:03:43,643 INFO [cloud.agent.AgentShell]
>> (main:null) Defaulting to using properties file for storage
>> 2016-08-18 14:03:43,645 INFO [cloud.agent.AgentShell]
>> (main:null) Defaulting to the constant time backoff algorithm
>> 2016-08-18 14:03:43,666 INFO [cloud.utils.LogUtils] (main:null)
>> log4j configuration found at
>> /usr/local/cloud/systemvm/conf/log4j-cloud.xml
>> 2016-08-18 14:03:43,686 INFO [cloud.agent.AgentShell]
>> (main:null) Using default Java settings for IPv6 preference for
>> agent connection
>> 2016-08-18 14:03:43,687 DEBUG [cloud.agent.AgentShell]
>> (main:null) Checking to see if agent.SecStorage.pid exists.
>> 2016-08-18 14:03:43,694 DEBUG [cloud.utils.ProcessUtil]
>> (main:null) environment.properties could not be opened
>> 2016-08-18 14:03:43,703 DEBUG [cloud.utils.ProcessUtil]
>> (main:null) Executing: bash -c echo $PPID
>> 2016-08-18 14:03:43,711 DEBUG [cloud.utils.ProcessUtil]
>> (main:null) Execution is successful.
>> 2016-08-18 14:03:43,726 ERROR [cloud.agent.AgentShell]
>> (main:null) Unable to start agent: Resource class not found:
>> com.cloud.storage.resource.PremiumSecondaryStorageResource due
>> to: java.lang.ClassNotFoundException:
>> com.cloud.storage.resource.PremiumSecondaryStorageResource
>>
>> ________________________________________
>> From: Tutkowski, Mike <Mi...@netapp.com>
>> Sent: Thursday, August 18, 2016 7:54 AM
>> To: dev@cloudstack.apache.org
>> Subject: Re: Issue with SSVM on KVM on master
>>
>> Thanks for the comments, Simon.
>>
>> I went ahead and tried to telnet into 8250 on my management
>> server (first from the SSVM and then from another machine). In
>> both cases, this was the result:
>>
>> telnet 192.168.129.50 8250
>> Trying 192.168.129.50...
>> Connected to 192.168.129.50.
>> Escape character is '^]'.
>> Connection closed by foreign host.
>>
>> It looks like a connection was established, but then shortly
>> after closed. Is this what you were expecting might happen?
>> ________________________________________
>> From: Simon Weller <sw...@ena.com>
>> Sent: Thursday, August 18, 2016 6:39 AM
>> To: dev@cloudstack.apache.org
>> Subject: Re: Issue with SSVM on KVM on master
>>
>> Mike,
>>
>> I wonder whether this could be related to the NIO connection
>> improvements. We saw this type of problem quite a bit when we
>> were testing it (with all types of agents). Typically you'll see
>> the agent logs indicating they can't connect to the management
>> server if this is the case. Try telnetting to 8250 on your mgmt
>> server and see if suddenly your agent starts to work.
>>
>> - Si
>>
>> ________________________________
>> From: Tutkowski, Mike <Mi...@netapp.com>
>> Sent: Wednesday, August 17, 2016 6:04 PM
>> To: dev@cloudstack.apache.org
>> Subject: Re: Issue with SSVM on KVM on master
>>
>> I should have pointed out that this is a Basic Zone. Also, there
>> is only one KVM host in my one cluster.
>> ________________________________________
>> From: Tutkowski, Mike <Mi...@netapp.com>
>> Sent: Wednesday, August 17, 2016 5:03 PM
>> To: dev@cloudstack.apache.org
>> Subject: Issue with SSVM on KVM on master
>>
>> Hi,
>>
>> I am trying to build a cloud making use of KVM system VMs.
>>
>> My SSVM VM comes up, but its agent doesn't seem to:
>>
>> http://imgur.com/a/Fo2ax
>> [http://i.imgur.com/xdP9alM.jpg?fb]<http://imgur.com/a/Fo2ax
>> ( http://i.imgur.com/xdP9alM.jpg?fb%5D<http://imgur.com/a/Fo2ax )>
>>
>> Imgur: The most awesome images on the
>> Internet<http://imgur.com/a/Fo2ax>
>> imgur.com
>> Imgur: The most awesome images on the Internet.
>>
>> Thoughts on what might be going on here?
>>
>> Thanks!
>>
>> Mike
>
>
Re: Issue with SSVM on KVM on master
Posted by "Tutkowski, Mike" <Mi...@netapp.com>.
As a little correction to what I said, I meant to ask if there is a way in 4.10 to run system VMs without using noredist for the build as I only have KVM hosts in this particular environment.
This does seem to work fine in 4.5, so it appears something has changed since that release in this regard.
> On Oct 11, 2016, at 12:15 AM, Tutkowski, Mike <Mi...@netapp.com> wrote:
>
> Hi Rajani,
>
> I actually ended up getting diverted to other work and am just coming back to this now. ☺
>
> Do you know if there’s a way in 4.10 to run the KVM agent without needing the noredist option? In this environment, I only have KVM hosts.
>
> I recently tried this in 4.5 and it worked fine without having to do anything special. It seems something has changed since 4.5 here.
>
> Thanks!
> Mike
>
> On 8/22/16, 12:43 AM, "Tutkowski, Mike" <Mi...@netapp.com> wrote:
>
> I can try that. Thanks, Rajani!
>
>> On Aug 22, 2016, at 12:17 AM, Rajani Karuturi <ra...@apache.org> wrote:
>>
>> Did you build and run it with noredist option?
>>
>> I see this CNF error when I forget to run jetty with noredist
>> option.
>>
>> ~ Rajani
>> http://cloudplatform.accelerite.com/
>>
>> On August 18, 2016 at 7:57 PM, Tutkowski, Mike
>> (Mike.Tutkowski@netapp.com) wrote:
>> Based on the error message of not being able to find the
>> PremiumSecondaryStorageResource class, this seems to be the
>> following error (#9 on this
>> page: https://cwiki.apache.org/confluence/display/CLOUDSTACK/SSVM,+templates,+Secondary+storage+troubleshooting):
>>
>> SSVM agent not running - You see error something like below. Most
>> probably your systemvm didnt get patched with the agent specific
>> code. This patching happens through the ISO - something
>> like systemvm***.iso. Check the size and location of the iso and
>> whether you are not using the old version iso. For XS its on the
>> host so grep for it and for vmware its on secondary storage. Do
>> also check that if its vmware that you built your ssvm using
>> noredist flag for mvn command. For more information check
>> SystemVm.iso
>> 2013-12-20 13:35:12,954 DEBUG [cloud.utils.ProcessUtil]
>> (main:null) Execution is successful.
>> 2013-12-20 13:35:12,960 ERROR [cloud.agent.AgentShell]
>> (main:null) Unable to start agent: Resource class not found:
>> com.cloud.storage.resource.PremiumSecondaryStorageResource due
>> to: java.lang.ClassNotFoundException:
>> com.cloud.storage.resource.PremiumSecondaryStorageResource
>>
>> I do see the ISO, though, on secondary storage here:
>>
>> cloudstack@cloudstack-virtual-machine:/export/secondary/systemvm$
>> ls -l
>> total 83420
>> -rw-rw-r-- 1 cloudstack cloudstack 85420032 Aug 15 16:10
>> systemvm-4.10.0-SNAPSHOT.iso
>> ________________________________________
>> From: Tutkowski, Mike <Mi...@netapp.com>
>> Sent: Thursday, August 18, 2016 8:17 AM
>> To: dev@cloudstack.apache.org
>> Subject: Re: Issue with SSVM on KVM on master
>>
>> In /var/log/cloud/cloud.out, I see the following repeated over
>> and over again (below).
>>
>> I'm running version 4.10.0 of the management server (but the
>> implementation version of the agent looks to be 4.9.0). I do have
>> systemvm-4.10.0-SNAPSHOT.iso accessible at
>> /export/secondary/systemvm (/export/secondary is my one and only
>> secondary storage at the time being).
>>
>> 2016-08-18 14:03:43,633 INFO [cloud.agent.AgentShell]
>> (main:null) Agent started
>> 2016-08-18 14:03:43,637 INFO [cloud.agent.AgentShell]
>> (main:null) Implementation Version is 4.9.0
>> 2016-08-18 14:03:43,637 INFO [cloud.agent.AgentShell]
>> (main:null) agent.properties found at
>> /usr/local/cloud/systemvm/conf/agent.properties
>> 2016-08-18 14:03:43,643 DEBUG [cloud.agent.AgentShell]
>> (main:null) Found property: instance
>> 2016-08-18 14:03:43,643 DEBUG [cloud.agent.AgentShell]
>> (main:null) Found property: resource
>> 2016-08-18 14:03:43,643 INFO [cloud.agent.AgentShell]
>> (main:null) Defaulting to using properties file for storage
>> 2016-08-18 14:03:43,645 INFO [cloud.agent.AgentShell]
>> (main:null) Defaulting to the constant time backoff algorithm
>> 2016-08-18 14:03:43,666 INFO [cloud.utils.LogUtils] (main:null)
>> log4j configuration found at
>> /usr/local/cloud/systemvm/conf/log4j-cloud.xml
>> 2016-08-18 14:03:43,686 INFO [cloud.agent.AgentShell]
>> (main:null) Using default Java settings for IPv6 preference for
>> agent connection
>> 2016-08-18 14:03:43,687 DEBUG [cloud.agent.AgentShell]
>> (main:null) Checking to see if agent.SecStorage.pid exists.
>> 2016-08-18 14:03:43,694 DEBUG [cloud.utils.ProcessUtil]
>> (main:null) environment.properties could not be opened
>> 2016-08-18 14:03:43,703 DEBUG [cloud.utils.ProcessUtil]
>> (main:null) Executing: bash -c echo $PPID
>> 2016-08-18 14:03:43,711 DEBUG [cloud.utils.ProcessUtil]
>> (main:null) Execution is successful.
>> 2016-08-18 14:03:43,726 ERROR [cloud.agent.AgentShell]
>> (main:null) Unable to start agent: Resource class not found:
>> com.cloud.storage.resource.PremiumSecondaryStorageResource due
>> to: java.lang.ClassNotFoundException:
>> com.cloud.storage.resource.PremiumSecondaryStorageResource
>>
>> ________________________________________
>> From: Tutkowski, Mike <Mi...@netapp.com>
>> Sent: Thursday, August 18, 2016 7:54 AM
>> To: dev@cloudstack.apache.org
>> Subject: Re: Issue with SSVM on KVM on master
>>
>> Thanks for the comments, Simon.
>>
>> I went ahead and tried to telnet into 8250 on my management
>> server (first from the SSVM and then from another machine). In
>> both cases, this was the result:
>>
>> telnet 192.168.129.50 8250
>> Trying 192.168.129.50...
>> Connected to 192.168.129.50.
>> Escape character is '^]'.
>> Connection closed by foreign host.
>>
>> It looks like a connection was established, but then shortly
>> after closed. Is this what you were expecting might happen?
>> ________________________________________
>> From: Simon Weller <sw...@ena.com>
>> Sent: Thursday, August 18, 2016 6:39 AM
>> To: dev@cloudstack.apache.org
>> Subject: Re: Issue with SSVM on KVM on master
>>
>> Mike,
>>
>> I wonder whether this could be related to the NIO connection
>> improvements. We saw this type of problem quite a bit when we
>> were testing it (with all types of agents). Typically you'll see
>> the agent logs indicating they can't connect to the management
>> server if this is the case. Try telnetting to 8250 on your mgmt
>> server and see if suddenly your agent starts to work.
>>
>> - Si
>>
>> ________________________________
>> From: Tutkowski, Mike <Mi...@netapp.com>
>> Sent: Wednesday, August 17, 2016 6:04 PM
>> To: dev@cloudstack.apache.org
>> Subject: Re: Issue with SSVM on KVM on master
>>
>> I should have pointed out that this is a Basic Zone. Also, there
>> is only one KVM host in my one cluster.
>> ________________________________________
>> From: Tutkowski, Mike <Mi...@netapp.com>
>> Sent: Wednesday, August 17, 2016 5:03 PM
>> To: dev@cloudstack.apache.org
>> Subject: Issue with SSVM on KVM on master
>>
>> Hi,
>>
>> I am trying to build a cloud making use of KVM system VMs.
>>
>> My SSVM VM comes up, but its agent doesn't seem to:
>>
>> http://imgur.com/a/Fo2ax
>> [http://i.imgur.com/xdP9alM.jpg?fb]<http://imgur.com/a/Fo2ax
>> ( http://i.imgur.com/xdP9alM.jpg?fb%5D<http://imgur.com/a/Fo2ax )>
>>
>> Imgur: The most awesome images on the
>> Internet<http://imgur.com/a/Fo2ax>
>> imgur.com
>> Imgur: The most awesome images on the Internet.
>>
>> Thoughts on what might be going on here?
>>
>> Thanks!
>>
>> Mike
>
>
Re: Issue with SSVM on KVM on master
Posted by "Tutkowski, Mike" <Mi...@netapp.com>.
Hi Rajani,
I actually ended up getting diverted to other work and am just coming back to this now. ☺
Do you know if there’s a way in 4.10 to run the KVM agent without needing the noredist option? In this environment, I only have KVM hosts.
I recently tried this in 4.5 and it worked fine without having to do anything special. It seems something has changed since 4.5 here.
Thanks!
Mike
On 8/22/16, 12:43 AM, "Tutkowski, Mike" <Mi...@netapp.com> wrote:
I can try that. Thanks, Rajani!
> On Aug 22, 2016, at 12:17 AM, Rajani Karuturi <ra...@apache.org> wrote:
>
> Did you build and run it with noredist option?
>
> I see this CNF error when I forget to run jetty with noredist
> option.
>
> ~ Rajani
> http://cloudplatform.accelerite.com/
>
> On August 18, 2016 at 7:57 PM, Tutkowski, Mike
> (Mike.Tutkowski@netapp.com) wrote:
> Based on the error message of not being able to find the
> PremiumSecondaryStorageResource class, this seems to be the
> following error (#9 on this
> page: https://cwiki.apache.org/confluence/display/CLOUDSTACK/SSVM,+templates,+Secondary+storage+troubleshooting):
>
> SSVM agent not running - You see error something like below. Most
> probably your systemvm didnt get patched with the agent specific
> code. This patching happens through the ISO - something
> like systemvm***.iso. Check the size and location of the iso and
> whether you are not using the old version iso. For XS its on the
> host so grep for it and for vmware its on secondary storage. Do
> also check that if its vmware that you built your ssvm using
> noredist flag for mvn command. For more information check
> SystemVm.iso
> 2013-12-20 13:35:12,954 DEBUG [cloud.utils.ProcessUtil]
> (main:null) Execution is successful.
> 2013-12-20 13:35:12,960 ERROR [cloud.agent.AgentShell]
> (main:null) Unable to start agent: Resource class not found:
> com.cloud.storage.resource.PremiumSecondaryStorageResource due
> to: java.lang.ClassNotFoundException:
> com.cloud.storage.resource.PremiumSecondaryStorageResource
>
> I do see the ISO, though, on secondary storage here:
>
> cloudstack@cloudstack-virtual-machine:/export/secondary/systemvm$
> ls -l
> total 83420
> -rw-rw-r-- 1 cloudstack cloudstack 85420032 Aug 15 16:10
> systemvm-4.10.0-SNAPSHOT.iso
> ________________________________________
> From: Tutkowski, Mike <Mi...@netapp.com>
> Sent: Thursday, August 18, 2016 8:17 AM
> To: dev@cloudstack.apache.org
> Subject: Re: Issue with SSVM on KVM on master
>
> In /var/log/cloud/cloud.out, I see the following repeated over
> and over again (below).
>
> I'm running version 4.10.0 of the management server (but the
> implementation version of the agent looks to be 4.9.0). I do have
> systemvm-4.10.0-SNAPSHOT.iso accessible at
> /export/secondary/systemvm (/export/secondary is my one and only
> secondary storage at the time being).
>
> 2016-08-18 14:03:43,633 INFO [cloud.agent.AgentShell]
> (main:null) Agent started
> 2016-08-18 14:03:43,637 INFO [cloud.agent.AgentShell]
> (main:null) Implementation Version is 4.9.0
> 2016-08-18 14:03:43,637 INFO [cloud.agent.AgentShell]
> (main:null) agent.properties found at
> /usr/local/cloud/systemvm/conf/agent.properties
> 2016-08-18 14:03:43,643 DEBUG [cloud.agent.AgentShell]
> (main:null) Found property: instance
> 2016-08-18 14:03:43,643 DEBUG [cloud.agent.AgentShell]
> (main:null) Found property: resource
> 2016-08-18 14:03:43,643 INFO [cloud.agent.AgentShell]
> (main:null) Defaulting to using properties file for storage
> 2016-08-18 14:03:43,645 INFO [cloud.agent.AgentShell]
> (main:null) Defaulting to the constant time backoff algorithm
> 2016-08-18 14:03:43,666 INFO [cloud.utils.LogUtils] (main:null)
> log4j configuration found at
> /usr/local/cloud/systemvm/conf/log4j-cloud.xml
> 2016-08-18 14:03:43,686 INFO [cloud.agent.AgentShell]
> (main:null) Using default Java settings for IPv6 preference for
> agent connection
> 2016-08-18 14:03:43,687 DEBUG [cloud.agent.AgentShell]
> (main:null) Checking to see if agent.SecStorage.pid exists.
> 2016-08-18 14:03:43,694 DEBUG [cloud.utils.ProcessUtil]
> (main:null) environment.properties could not be opened
> 2016-08-18 14:03:43,703 DEBUG [cloud.utils.ProcessUtil]
> (main:null) Executing: bash -c echo $PPID
> 2016-08-18 14:03:43,711 DEBUG [cloud.utils.ProcessUtil]
> (main:null) Execution is successful.
> 2016-08-18 14:03:43,726 ERROR [cloud.agent.AgentShell]
> (main:null) Unable to start agent: Resource class not found:
> com.cloud.storage.resource.PremiumSecondaryStorageResource due
> to: java.lang.ClassNotFoundException:
> com.cloud.storage.resource.PremiumSecondaryStorageResource
>
> ________________________________________
> From: Tutkowski, Mike <Mi...@netapp.com>
> Sent: Thursday, August 18, 2016 7:54 AM
> To: dev@cloudstack.apache.org
> Subject: Re: Issue with SSVM on KVM on master
>
> Thanks for the comments, Simon.
>
> I went ahead and tried to telnet into 8250 on my management
> server (first from the SSVM and then from another machine). In
> both cases, this was the result:
>
> telnet 192.168.129.50 8250
> Trying 192.168.129.50...
> Connected to 192.168.129.50.
> Escape character is '^]'.
> Connection closed by foreign host.
>
> It looks like a connection was established, but then shortly
> after closed. Is this what you were expecting might happen?
> ________________________________________
> From: Simon Weller <sw...@ena.com>
> Sent: Thursday, August 18, 2016 6:39 AM
> To: dev@cloudstack.apache.org
> Subject: Re: Issue with SSVM on KVM on master
>
> Mike,
>
> I wonder whether this could be related to the NIO connection
> improvements. We saw this type of problem quite a bit when we
> were testing it (with all types of agents). Typically you'll see
> the agent logs indicating they can't connect to the management
> server if this is the case. Try telnetting to 8250 on your mgmt
> server and see if suddenly your agent starts to work.
>
> - Si
>
> ________________________________
> From: Tutkowski, Mike <Mi...@netapp.com>
> Sent: Wednesday, August 17, 2016 6:04 PM
> To: dev@cloudstack.apache.org
> Subject: Re: Issue with SSVM on KVM on master
>
> I should have pointed out that this is a Basic Zone. Also, there
> is only one KVM host in my one cluster.
> ________________________________________
> From: Tutkowski, Mike <Mi...@netapp.com>
> Sent: Wednesday, August 17, 2016 5:03 PM
> To: dev@cloudstack.apache.org
> Subject: Issue with SSVM on KVM on master
>
> Hi,
>
> I am trying to build a cloud making use of KVM system VMs.
>
> My SSVM VM comes up, but its agent doesn't seem to:
>
> http://imgur.com/a/Fo2ax
> [http://i.imgur.com/xdP9alM.jpg?fb]<http://imgur.com/a/Fo2ax
> ( http://i.imgur.com/xdP9alM.jpg?fb%5D<http://imgur.com/a/Fo2ax )>
>
> Imgur: The most awesome images on the
> Internet<http://imgur.com/a/Fo2ax>
> imgur.com
> Imgur: The most awesome images on the Internet.
>
> Thoughts on what might be going on here?
>
> Thanks!
>
> Mike
Re: Issue with SSVM on KVM on master
Posted by "Tutkowski, Mike" <Mi...@netapp.com>.
I can try that. Thanks, Rajani!
> On Aug 22, 2016, at 12:17 AM, Rajani Karuturi <ra...@apache.org> wrote:
>
> Did you build and run it with noredist option?
>
> I see this CNF error when I forget to run jetty with noredist
> option.
>
> ~ Rajani
> http://cloudplatform.accelerite.com/
>
> On August 18, 2016 at 7:57 PM, Tutkowski, Mike
> (Mike.Tutkowski@netapp.com) wrote:
> Based on the error message of not being able to find the
> PremiumSecondaryStorageResource class, this seems to be the
> following error (#9 on this
> page: https://cwiki.apache.org/confluence/display/CLOUDSTACK/SSVM,+templates,+Secondary+storage+troubleshooting):
>
> SSVM agent not running - You see error something like below. Most
> probably your systemvm didnt get patched with the agent specific
> code. This patching happens through the ISO - something
> like systemvm***.iso. Check the size and location of the iso and
> whether you are not using the old version iso. For XS its on the
> host so grep for it and for vmware its on secondary storage. Do
> also check that if its vmware that you built your ssvm using
> noredist flag for mvn command. For more information check
> SystemVm.iso
> 2013-12-20 13:35:12,954 DEBUG [cloud.utils.ProcessUtil]
> (main:null) Execution is successful.
> 2013-12-20 13:35:12,960 ERROR [cloud.agent.AgentShell]
> (main:null) Unable to start agent: Resource class not found:
> com.cloud.storage.resource.PremiumSecondaryStorageResource due
> to: java.lang.ClassNotFoundException:
> com.cloud.storage.resource.PremiumSecondaryStorageResource
>
> I do see the ISO, though, on secondary storage here:
>
> cloudstack@cloudstack-virtual-machine:/export/secondary/systemvm$
> ls -l
> total 83420
> -rw-rw-r-- 1 cloudstack cloudstack 85420032 Aug 15 16:10
> systemvm-4.10.0-SNAPSHOT.iso
> ________________________________________
> From: Tutkowski, Mike <Mi...@netapp.com>
> Sent: Thursday, August 18, 2016 8:17 AM
> To: dev@cloudstack.apache.org
> Subject: Re: Issue with SSVM on KVM on master
>
> In /var/log/cloud/cloud.out, I see the following repeated over
> and over again (below).
>
> I'm running version 4.10.0 of the management server (but the
> implementation version of the agent looks to be 4.9.0). I do have
> systemvm-4.10.0-SNAPSHOT.iso accessible at
> /export/secondary/systemvm (/export/secondary is my one and only
> secondary storage at the time being).
>
> 2016-08-18 14:03:43,633 INFO [cloud.agent.AgentShell]
> (main:null) Agent started
> 2016-08-18 14:03:43,637 INFO [cloud.agent.AgentShell]
> (main:null) Implementation Version is 4.9.0
> 2016-08-18 14:03:43,637 INFO [cloud.agent.AgentShell]
> (main:null) agent.properties found at
> /usr/local/cloud/systemvm/conf/agent.properties
> 2016-08-18 14:03:43,643 DEBUG [cloud.agent.AgentShell]
> (main:null) Found property: instance
> 2016-08-18 14:03:43,643 DEBUG [cloud.agent.AgentShell]
> (main:null) Found property: resource
> 2016-08-18 14:03:43,643 INFO [cloud.agent.AgentShell]
> (main:null) Defaulting to using properties file for storage
> 2016-08-18 14:03:43,645 INFO [cloud.agent.AgentShell]
> (main:null) Defaulting to the constant time backoff algorithm
> 2016-08-18 14:03:43,666 INFO [cloud.utils.LogUtils] (main:null)
> log4j configuration found at
> /usr/local/cloud/systemvm/conf/log4j-cloud.xml
> 2016-08-18 14:03:43,686 INFO [cloud.agent.AgentShell]
> (main:null) Using default Java settings for IPv6 preference for
> agent connection
> 2016-08-18 14:03:43,687 DEBUG [cloud.agent.AgentShell]
> (main:null) Checking to see if agent.SecStorage.pid exists.
> 2016-08-18 14:03:43,694 DEBUG [cloud.utils.ProcessUtil]
> (main:null) environment.properties could not be opened
> 2016-08-18 14:03:43,703 DEBUG [cloud.utils.ProcessUtil]
> (main:null) Executing: bash -c echo $PPID
> 2016-08-18 14:03:43,711 DEBUG [cloud.utils.ProcessUtil]
> (main:null) Execution is successful.
> 2016-08-18 14:03:43,726 ERROR [cloud.agent.AgentShell]
> (main:null) Unable to start agent: Resource class not found:
> com.cloud.storage.resource.PremiumSecondaryStorageResource due
> to: java.lang.ClassNotFoundException:
> com.cloud.storage.resource.PremiumSecondaryStorageResource
>
> ________________________________________
> From: Tutkowski, Mike <Mi...@netapp.com>
> Sent: Thursday, August 18, 2016 7:54 AM
> To: dev@cloudstack.apache.org
> Subject: Re: Issue with SSVM on KVM on master
>
> Thanks for the comments, Simon.
>
> I went ahead and tried to telnet into 8250 on my management
> server (first from the SSVM and then from another machine). In
> both cases, this was the result:
>
> telnet 192.168.129.50 8250
> Trying 192.168.129.50...
> Connected to 192.168.129.50.
> Escape character is '^]'.
> Connection closed by foreign host.
>
> It looks like a connection was established, but then shortly
> after closed. Is this what you were expecting might happen?
> ________________________________________
> From: Simon Weller <sw...@ena.com>
> Sent: Thursday, August 18, 2016 6:39 AM
> To: dev@cloudstack.apache.org
> Subject: Re: Issue with SSVM on KVM on master
>
> Mike,
>
> I wonder whether this could be related to the NIO connection
> improvements. We saw this type of problem quite a bit when we
> were testing it (with all types of agents). Typically you'll see
> the agent logs indicating they can't connect to the management
> server if this is the case. Try telnetting to 8250 on your mgmt
> server and see if suddenly your agent starts to work.
>
> - Si
>
> ________________________________
> From: Tutkowski, Mike <Mi...@netapp.com>
> Sent: Wednesday, August 17, 2016 6:04 PM
> To: dev@cloudstack.apache.org
> Subject: Re: Issue with SSVM on KVM on master
>
> I should have pointed out that this is a Basic Zone. Also, there
> is only one KVM host in my one cluster.
> ________________________________________
> From: Tutkowski, Mike <Mi...@netapp.com>
> Sent: Wednesday, August 17, 2016 5:03 PM
> To: dev@cloudstack.apache.org
> Subject: Issue with SSVM on KVM on master
>
> Hi,
>
> I am trying to build a cloud making use of KVM system VMs.
>
> My SSVM VM comes up, but its agent doesn't seem to:
>
> http://imgur.com/a/Fo2ax
> [http://i.imgur.com/xdP9alM.jpg?fb]<http://imgur.com/a/Fo2ax
> ( http://i.imgur.com/xdP9alM.jpg?fb%5D<http://imgur.com/a/Fo2ax )>
>
> Imgur: The most awesome images on the
> Internet<http://imgur.com/a/Fo2ax>
> imgur.com
> Imgur: The most awesome images on the Internet.
>
> Thoughts on what might be going on here?
>
> Thanks!
>
> Mike
Re: Issue with SSVM on KVM on master
Posted by Rajani Karuturi <ra...@apache.org>.
Did you build and run it with noredist option?
I see this CNF error when I forget to run jetty with noredist
option.
~ Rajani
http://cloudplatform.accelerite.com/
On August 18, 2016 at 7:57 PM, Tutkowski, Mike
(Mike.Tutkowski@netapp.com) wrote:
Based on the error message of not being able to find the
PremiumSecondaryStorageResource class, this seems to be the
following error (#9 on this
page: https://cwiki.apache.org/confluence/display/CLOUDSTACK/SSVM,+templates,+Secondary+storage+troubleshooting):
SSVM agent not running - You see error something like below. Most
probably your systemvm didnt get patched with the agent specific
code. This patching happens through the ISO - something
like systemvm***.iso. Check the size and location of the iso and
whether you are not using the old version iso. For XS its on the
host so grep for it and for vmware its on secondary storage. Do
also check that if its vmware that you built your ssvm using
noredist flag for mvn command. For more information check
SystemVm.iso
2013-12-20 13:35:12,954 DEBUG [cloud.utils.ProcessUtil]
(main:null) Execution is successful.
2013-12-20 13:35:12,960 ERROR [cloud.agent.AgentShell]
(main:null) Unable to start agent: Resource class not found:
com.cloud.storage.resource.PremiumSecondaryStorageResource due
to: java.lang.ClassNotFoundException:
com.cloud.storage.resource.PremiumSecondaryStorageResource
I do see the ISO, though, on secondary storage here:
cloudstack@cloudstack-virtual-machine:/export/secondary/systemvm$
ls -l
total 83420
-rw-rw-r-- 1 cloudstack cloudstack 85420032 Aug 15 16:10
systemvm-4.10.0-SNAPSHOT.iso
________________________________________
From: Tutkowski, Mike <Mi...@netapp.com>
Sent: Thursday, August 18, 2016 8:17 AM
To: dev@cloudstack.apache.org
Subject: Re: Issue with SSVM on KVM on master
In /var/log/cloud/cloud.out, I see the following repeated over
and over again (below).
I'm running version 4.10.0 of the management server (but the
implementation version of the agent looks to be 4.9.0). I do have
systemvm-4.10.0-SNAPSHOT.iso accessible at
/export/secondary/systemvm (/export/secondary is my one and only
secondary storage at the time being).
2016-08-18 14:03:43,633 INFO [cloud.agent.AgentShell]
(main:null) Agent started
2016-08-18 14:03:43,637 INFO [cloud.agent.AgentShell]
(main:null) Implementation Version is 4.9.0
2016-08-18 14:03:43,637 INFO [cloud.agent.AgentShell]
(main:null) agent.properties found at
/usr/local/cloud/systemvm/conf/agent.properties
2016-08-18 14:03:43,643 DEBUG [cloud.agent.AgentShell]
(main:null) Found property: instance
2016-08-18 14:03:43,643 DEBUG [cloud.agent.AgentShell]
(main:null) Found property: resource
2016-08-18 14:03:43,643 INFO [cloud.agent.AgentShell]
(main:null) Defaulting to using properties file for storage
2016-08-18 14:03:43,645 INFO [cloud.agent.AgentShell]
(main:null) Defaulting to the constant time backoff algorithm
2016-08-18 14:03:43,666 INFO [cloud.utils.LogUtils] (main:null)
log4j configuration found at
/usr/local/cloud/systemvm/conf/log4j-cloud.xml
2016-08-18 14:03:43,686 INFO [cloud.agent.AgentShell]
(main:null) Using default Java settings for IPv6 preference for
agent connection
2016-08-18 14:03:43,687 DEBUG [cloud.agent.AgentShell]
(main:null) Checking to see if agent.SecStorage.pid exists.
2016-08-18 14:03:43,694 DEBUG [cloud.utils.ProcessUtil]
(main:null) environment.properties could not be opened
2016-08-18 14:03:43,703 DEBUG [cloud.utils.ProcessUtil]
(main:null) Executing: bash -c echo $PPID
2016-08-18 14:03:43,711 DEBUG [cloud.utils.ProcessUtil]
(main:null) Execution is successful.
2016-08-18 14:03:43,726 ERROR [cloud.agent.AgentShell]
(main:null) Unable to start agent: Resource class not found:
com.cloud.storage.resource.PremiumSecondaryStorageResource due
to: java.lang.ClassNotFoundException:
com.cloud.storage.resource.PremiumSecondaryStorageResource
________________________________________
From: Tutkowski, Mike <Mi...@netapp.com>
Sent: Thursday, August 18, 2016 7:54 AM
To: dev@cloudstack.apache.org
Subject: Re: Issue with SSVM on KVM on master
Thanks for the comments, Simon.
I went ahead and tried to telnet into 8250 on my management
server (first from the SSVM and then from another machine). In
both cases, this was the result:
telnet 192.168.129.50 8250
Trying 192.168.129.50...
Connected to 192.168.129.50.
Escape character is '^]'.
Connection closed by foreign host.
It looks like a connection was established, but then shortly
after closed. Is this what you were expecting might happen?
________________________________________
From: Simon Weller <sw...@ena.com>
Sent: Thursday, August 18, 2016 6:39 AM
To: dev@cloudstack.apache.org
Subject: Re: Issue with SSVM on KVM on master
Mike,
I wonder whether this could be related to the NIO connection
improvements. We saw this type of problem quite a bit when we
were testing it (with all types of agents). Typically you'll see
the agent logs indicating they can't connect to the management
server if this is the case. Try telnetting to 8250 on your mgmt
server and see if suddenly your agent starts to work.
- Si
________________________________
From: Tutkowski, Mike <Mi...@netapp.com>
Sent: Wednesday, August 17, 2016 6:04 PM
To: dev@cloudstack.apache.org
Subject: Re: Issue with SSVM on KVM on master
I should have pointed out that this is a Basic Zone. Also, there
is only one KVM host in my one cluster.
________________________________________
From: Tutkowski, Mike <Mi...@netapp.com>
Sent: Wednesday, August 17, 2016 5:03 PM
To: dev@cloudstack.apache.org
Subject: Issue with SSVM on KVM on master
Hi,
I am trying to build a cloud making use of KVM system VMs.
My SSVM VM comes up, but its agent doesn't seem to:
http://imgur.com/a/Fo2ax
[http://i.imgur.com/xdP9alM.jpg?fb]<http://imgur.com/a/Fo2ax
( http://i.imgur.com/xdP9alM.jpg?fb%5D<http://imgur.com/a/Fo2ax )>
Imgur: The most awesome images on the
Internet<http://imgur.com/a/Fo2ax>
imgur.com
Imgur: The most awesome images on the Internet.
Thoughts on what might be going on here?
Thanks!
Mike
Re: Issue with SSVM on KVM on master
Posted by "Tutkowski, Mike" <Mi...@netapp.com>.
Based on the error message of not being able to find the PremiumSecondaryStorageResource class, this seems to be the following error (#9 on this page: https://cwiki.apache.org/confluence/display/CLOUDSTACK/SSVM,+templates,+Secondary+storage+troubleshooting):
SSVM agent not running - You see error something like below. Most probably your systemvm didnt get patched with the agent specific code. This patching happens through the ISO - something like systemvm***.iso. Check the size and location of the iso and whether you are not using the old version iso. For XS its on the host so grep for it and for vmware its on secondary storage. Do also check that if its vmware that you built your ssvm using noredist flag for mvn command. For more information check SystemVm.iso
2013-12-20 13:35:12,954 DEBUG [cloud.utils.ProcessUtil] (main:null) Execution is successful.
2013-12-20 13:35:12,960 ERROR [cloud.agent.AgentShell] (main:null) Unable to start agent: Resource class not found: com.cloud.storage.resource.PremiumSecondaryStorageResource due to: java.lang.ClassNotFoundException: com.cloud.storage.resource.PremiumSecondaryStorageResource
I do see the ISO, though, on secondary storage here:
cloudstack@cloudstack-virtual-machine:/export/secondary/systemvm$ ls -l
total 83420
-rw-rw-r-- 1 cloudstack cloudstack 85420032 Aug 15 16:10 systemvm-4.10.0-SNAPSHOT.iso
________________________________________
From: Tutkowski, Mike <Mi...@netapp.com>
Sent: Thursday, August 18, 2016 8:17 AM
To: dev@cloudstack.apache.org
Subject: Re: Issue with SSVM on KVM on master
In /var/log/cloud/cloud.out, I see the following repeated over and over again (below).
I'm running version 4.10.0 of the management server (but the implementation version of the agent looks to be 4.9.0). I do have systemvm-4.10.0-SNAPSHOT.iso accessible at /export/secondary/systemvm (/export/secondary is my one and only secondary storage at the time being).
2016-08-18 14:03:43,633 INFO [cloud.agent.AgentShell] (main:null) Agent started
2016-08-18 14:03:43,637 INFO [cloud.agent.AgentShell] (main:null) Implementation Version is 4.9.0
2016-08-18 14:03:43,637 INFO [cloud.agent.AgentShell] (main:null) agent.properties found at /usr/local/cloud/systemvm/conf/agent.properties
2016-08-18 14:03:43,643 DEBUG [cloud.agent.AgentShell] (main:null) Found property: instance
2016-08-18 14:03:43,643 DEBUG [cloud.agent.AgentShell] (main:null) Found property: resource
2016-08-18 14:03:43,643 INFO [cloud.agent.AgentShell] (main:null) Defaulting to using properties file for storage
2016-08-18 14:03:43,645 INFO [cloud.agent.AgentShell] (main:null) Defaulting to the constant time backoff algorithm
2016-08-18 14:03:43,666 INFO [cloud.utils.LogUtils] (main:null) log4j configuration found at /usr/local/cloud/systemvm/conf/log4j-cloud.xml
2016-08-18 14:03:43,686 INFO [cloud.agent.AgentShell] (main:null) Using default Java settings for IPv6 preference for agent connection
2016-08-18 14:03:43,687 DEBUG [cloud.agent.AgentShell] (main:null) Checking to see if agent.SecStorage.pid exists.
2016-08-18 14:03:43,694 DEBUG [cloud.utils.ProcessUtil] (main:null) environment.properties could not be opened
2016-08-18 14:03:43,703 DEBUG [cloud.utils.ProcessUtil] (main:null) Executing: bash -c echo $PPID
2016-08-18 14:03:43,711 DEBUG [cloud.utils.ProcessUtil] (main:null) Execution is successful.
2016-08-18 14:03:43,726 ERROR [cloud.agent.AgentShell] (main:null) Unable to start agent: Resource class not found: com.cloud.storage.resource.PremiumSecondaryStorageResource due to: java.lang.ClassNotFoundException: com.cloud.storage.resource.PremiumSecondaryStorageResource
________________________________________
From: Tutkowski, Mike <Mi...@netapp.com>
Sent: Thursday, August 18, 2016 7:54 AM
To: dev@cloudstack.apache.org
Subject: Re: Issue with SSVM on KVM on master
Thanks for the comments, Simon.
I went ahead and tried to telnet into 8250 on my management server (first from the SSVM and then from another machine). In both cases, this was the result:
telnet 192.168.129.50 8250
Trying 192.168.129.50...
Connected to 192.168.129.50.
Escape character is '^]'.
Connection closed by foreign host.
It looks like a connection was established, but then shortly after closed. Is this what you were expecting might happen?
________________________________________
From: Simon Weller <sw...@ena.com>
Sent: Thursday, August 18, 2016 6:39 AM
To: dev@cloudstack.apache.org
Subject: Re: Issue with SSVM on KVM on master
Mike,
I wonder whether this could be related to the NIO connection improvements. We saw this type of problem quite a bit when we were testing it (with all types of agents). Typically you'll see the agent logs indicating they can't connect to the management server if this is the case. Try telnetting to 8250 on your mgmt server and see if suddenly your agent starts to work.
- Si
________________________________
From: Tutkowski, Mike <Mi...@netapp.com>
Sent: Wednesday, August 17, 2016 6:04 PM
To: dev@cloudstack.apache.org
Subject: Re: Issue with SSVM on KVM on master
I should have pointed out that this is a Basic Zone. Also, there is only one KVM host in my one cluster.
________________________________________
From: Tutkowski, Mike <Mi...@netapp.com>
Sent: Wednesday, August 17, 2016 5:03 PM
To: dev@cloudstack.apache.org
Subject: Issue with SSVM on KVM on master
Hi,
I am trying to build a cloud making use of KVM system VMs.
My SSVM VM comes up, but its agent doesn't seem to:
http://imgur.com/a/Fo2ax
[http://i.imgur.com/xdP9alM.jpg?fb]<http://imgur.com/a/Fo2ax>
Imgur: The most awesome images on the Internet<http://imgur.com/a/Fo2ax>
imgur.com
Imgur: The most awesome images on the Internet.
Thoughts on what might be going on here?
Thanks!
Mike
Re: Issue with SSVM on KVM on master
Posted by "Tutkowski, Mike" <Mi...@netapp.com>.
In /var/log/cloud/cloud.out, I see the following repeated over and over again (below).
I'm running version 4.10.0 of the management server (but the implementation version of the agent looks to be 4.9.0). I do have systemvm-4.10.0-SNAPSHOT.iso accessible at /export/secondary/systemvm (/export/secondary is my one and only secondary storage at the time being).
2016-08-18 14:03:43,633 INFO [cloud.agent.AgentShell] (main:null) Agent started
2016-08-18 14:03:43,637 INFO [cloud.agent.AgentShell] (main:null) Implementation Version is 4.9.0
2016-08-18 14:03:43,637 INFO [cloud.agent.AgentShell] (main:null) agent.properties found at /usr/local/cloud/systemvm/conf/agent.properties
2016-08-18 14:03:43,643 DEBUG [cloud.agent.AgentShell] (main:null) Found property: instance
2016-08-18 14:03:43,643 DEBUG [cloud.agent.AgentShell] (main:null) Found property: resource
2016-08-18 14:03:43,643 INFO [cloud.agent.AgentShell] (main:null) Defaulting to using properties file for storage
2016-08-18 14:03:43,645 INFO [cloud.agent.AgentShell] (main:null) Defaulting to the constant time backoff algorithm
2016-08-18 14:03:43,666 INFO [cloud.utils.LogUtils] (main:null) log4j configuration found at /usr/local/cloud/systemvm/conf/log4j-cloud.xml
2016-08-18 14:03:43,686 INFO [cloud.agent.AgentShell] (main:null) Using default Java settings for IPv6 preference for agent connection
2016-08-18 14:03:43,687 DEBUG [cloud.agent.AgentShell] (main:null) Checking to see if agent.SecStorage.pid exists.
2016-08-18 14:03:43,694 DEBUG [cloud.utils.ProcessUtil] (main:null) environment.properties could not be opened
2016-08-18 14:03:43,703 DEBUG [cloud.utils.ProcessUtil] (main:null) Executing: bash -c echo $PPID
2016-08-18 14:03:43,711 DEBUG [cloud.utils.ProcessUtil] (main:null) Execution is successful.
2016-08-18 14:03:43,726 ERROR [cloud.agent.AgentShell] (main:null) Unable to start agent: Resource class not found: com.cloud.storage.resource.PremiumSecondaryStorageResource due to: java.lang.ClassNotFoundException: com.cloud.storage.resource.PremiumSecondaryStorageResource
________________________________________
From: Tutkowski, Mike <Mi...@netapp.com>
Sent: Thursday, August 18, 2016 7:54 AM
To: dev@cloudstack.apache.org
Subject: Re: Issue with SSVM on KVM on master
Thanks for the comments, Simon.
I went ahead and tried to telnet into 8250 on my management server (first from the SSVM and then from another machine). In both cases, this was the result:
telnet 192.168.129.50 8250
Trying 192.168.129.50...
Connected to 192.168.129.50.
Escape character is '^]'.
Connection closed by foreign host.
It looks like a connection was established, but then shortly after closed. Is this what you were expecting might happen?
________________________________________
From: Simon Weller <sw...@ena.com>
Sent: Thursday, August 18, 2016 6:39 AM
To: dev@cloudstack.apache.org
Subject: Re: Issue with SSVM on KVM on master
Mike,
I wonder whether this could be related to the NIO connection improvements. We saw this type of problem quite a bit when we were testing it (with all types of agents). Typically you'll see the agent logs indicating they can't connect to the management server if this is the case. Try telnetting to 8250 on your mgmt server and see if suddenly your agent starts to work.
- Si
________________________________
From: Tutkowski, Mike <Mi...@netapp.com>
Sent: Wednesday, August 17, 2016 6:04 PM
To: dev@cloudstack.apache.org
Subject: Re: Issue with SSVM on KVM on master
I should have pointed out that this is a Basic Zone. Also, there is only one KVM host in my one cluster.
________________________________________
From: Tutkowski, Mike <Mi...@netapp.com>
Sent: Wednesday, August 17, 2016 5:03 PM
To: dev@cloudstack.apache.org
Subject: Issue with SSVM on KVM on master
Hi,
I am trying to build a cloud making use of KVM system VMs.
My SSVM VM comes up, but its agent doesn't seem to:
http://imgur.com/a/Fo2ax
[http://i.imgur.com/xdP9alM.jpg?fb]<http://imgur.com/a/Fo2ax>
Imgur: The most awesome images on the Internet<http://imgur.com/a/Fo2ax>
imgur.com
Imgur: The most awesome images on the Internet.
Thoughts on what might be going on here?
Thanks!
Mike
Re: Issue with SSVM on KVM on master
Posted by "Tutkowski, Mike" <Mi...@netapp.com>.
Thanks for the comments, Simon.
I went ahead and tried to telnet into 8250 on my management server (first from the SSVM and then from another machine). In both cases, this was the result:
telnet 192.168.129.50 8250
Trying 192.168.129.50...
Connected to 192.168.129.50.
Escape character is '^]'.
Connection closed by foreign host.
It looks like a connection was established, but then shortly after closed. Is this what you were expecting might happen?
________________________________________
From: Simon Weller <sw...@ena.com>
Sent: Thursday, August 18, 2016 6:39 AM
To: dev@cloudstack.apache.org
Subject: Re: Issue with SSVM on KVM on master
Mike,
I wonder whether this could be related to the NIO connection improvements. We saw this type of problem quite a bit when we were testing it (with all types of agents). Typically you'll see the agent logs indicating they can't connect to the management server if this is the case. Try telnetting to 8250 on your mgmt server and see if suddenly your agent starts to work.
- Si
________________________________
From: Tutkowski, Mike <Mi...@netapp.com>
Sent: Wednesday, August 17, 2016 6:04 PM
To: dev@cloudstack.apache.org
Subject: Re: Issue with SSVM on KVM on master
I should have pointed out that this is a Basic Zone. Also, there is only one KVM host in my one cluster.
________________________________________
From: Tutkowski, Mike <Mi...@netapp.com>
Sent: Wednesday, August 17, 2016 5:03 PM
To: dev@cloudstack.apache.org
Subject: Issue with SSVM on KVM on master
Hi,
I am trying to build a cloud making use of KVM system VMs.
My SSVM VM comes up, but its agent doesn't seem to:
http://imgur.com/a/Fo2ax
[http://i.imgur.com/xdP9alM.jpg?fb]<http://imgur.com/a/Fo2ax>
Imgur: The most awesome images on the Internet<http://imgur.com/a/Fo2ax>
imgur.com
Imgur: The most awesome images on the Internet.
Thoughts on what might be going on here?
Thanks!
Mike
Re: Issue with SSVM on KVM on master
Posted by Simon Weller <sw...@ena.com>.
Mike,
I wonder whether this could be related to the NIO connection improvements. We saw this type of problem quite a bit when we were testing it (with all types of agents). Typically you'll see the agent logs indicating they can't connect to the management server if this is the case. Try telnetting to 8250 on your mgmt server and see if suddenly your agent starts to work.
- Si
________________________________
From: Tutkowski, Mike <Mi...@netapp.com>
Sent: Wednesday, August 17, 2016 6:04 PM
To: dev@cloudstack.apache.org
Subject: Re: Issue with SSVM on KVM on master
I should have pointed out that this is a Basic Zone. Also, there is only one KVM host in my one cluster.
________________________________________
From: Tutkowski, Mike <Mi...@netapp.com>
Sent: Wednesday, August 17, 2016 5:03 PM
To: dev@cloudstack.apache.org
Subject: Issue with SSVM on KVM on master
Hi,
I am trying to build a cloud making use of KVM system VMs.
My SSVM VM comes up, but its agent doesn't seem to:
http://imgur.com/a/Fo2ax
[http://i.imgur.com/xdP9alM.jpg?fb]<http://imgur.com/a/Fo2ax>
Imgur: The most awesome images on the Internet<http://imgur.com/a/Fo2ax>
imgur.com
Imgur: The most awesome images on the Internet.
Thoughts on what might be going on here?
Thanks!
Mike
Re: Issue with SSVM on KVM on master
Posted by "Tutkowski, Mike" <Mi...@netapp.com>.
I should have pointed out that this is a Basic Zone. Also, there is only one KVM host in my one cluster.
________________________________________
From: Tutkowski, Mike <Mi...@netapp.com>
Sent: Wednesday, August 17, 2016 5:03 PM
To: dev@cloudstack.apache.org
Subject: Issue with SSVM on KVM on master
Hi,
I am trying to build a cloud making use of KVM system VMs.
My SSVM VM comes up, but its agent doesn't seem to:
http://imgur.com/a/Fo2ax
Thoughts on what might be going on here?
Thanks!
Mike