You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@cloudstack.apache.org by Luciano Castro <lu...@gmail.com> on 2015/07/15 20:46:01 UTC
Fwd: HA feature - KVM - CloudStack 4.5.1
Hi there,
I´m new in the list, and I´m using 4.5.1 CloudStack version, and I
started making some tests with H.A feature for VMs,
but I didn´t have success. I don´t know if it´s a bug, or something that
I´m doing wrong.
Scenario: I have 1 manager server and 2 KVM hosts ('A' and 'B').
1) I created a "service offering" (computing offering) with the TAG H.A
checked.
2) then, I created a VM with this offering, and the VM is in KVM 'A'.
3) I did a shutdown in the KVM host 'A', and I expected that this VM, after
some minutes will be available in another KVM host ('B')
But it didn´t happened.
Any ideias??
thanks!
--
Luciano Castro
--
Luciano Castro
Re: HA feature - KVM - CloudStack 4.5.1
Posted by Anshul Gangwar <an...@citrix.com>.
VM status is still running because we cannot determine the status of VM on host which is in disconnected state. Host is in disconnected state may be due to network glitch but may be VM is still running on it.
Regards,
Anshul
On 16-Jul-2015, at 6:04 pm, Luciano Castro <lu...@gmail.com>> wrote:
Hi!
* Yes, Offer HA.
* The status os KVM hosts reported is "Disconnected", and curiously the VM
status is still "Running..."
I did the same test again:
1) shutdown KVM host 12:13 UTC
2) Logs (management server)
[root@1q2 ~]# grep -i -E 'warn|exception|unable|fail|error'
/var/log/cloudstack/management/management-server.log|tail -50
2015-07-16 12:13:21,195 WARN [c.c.a.d.ParamGenericValidationWorker]
(catalina-exec-14:ctx-b1ba1de6 ctx-5c5114f1) Received unknown parameters
for command listNetworkOfferings. Unknown parameters : projectid
2015-07-16 12:13:31,395 WARN [c.c.a.d.ParamGenericValidationWorker]
(catalina-exec-4:ctx-78cb8329 ctx-ea77f742) Received unknown parameters for
command listNetworkOfferings. Unknown parameters : projectid
2015-07-16 12:13:40,975 WARN [c.c.a.d.ParamGenericValidationWorker]
(catalina-exec-19:ctx-6e2ab11e ctx-f0bc3e8d) Received unknown parameters
for command expungeVirtualMachine. Unknown parameters : projectid
2015-07-16 12:13:40,998 WARN [c.c.a.d.ParamGenericValidationWorker]
(API-Job-Executor-54:ctx-a6765444 job-407 ctx-e7d4ae6e) Received unknown
parameters for command expungeVirtualMachine. Unknown parameters : projectid
2015-07-16 12:13:44,162 WARN [c.c.a.d.ParamGenericValidationWorker]
(catalina-exec-3:ctx-ca71fefa ctx-3a46d16f) Received unknown parameters for
command queryAsyncJobResult. Unknown parameters : projectid
2015-07-16 12:13:47,163 WARN [c.c.a.d.ParamGenericValidationWorker]
(catalina-exec-2:ctx-9773fe5c ctx-20f2c53d) Received unknown parameters for
command listNetworkOfferings. Unknown parameters : projectid
2015-07-16 12:13:51,478 WARN [c.c.a.d.ParamGenericValidationWorker]
(catalina-exec-4:ctx-986e5695 ctx-cfd9dc2a) Received unknown parameters for
command listNetworkOfferings. Unknown parameters : projectid
2015-07-16 12:13:57,139 WARN [c.c.a.d.ParamGenericValidationWorker]
(catalina-exec-21:ctx-3ed5162d ctx-0b378da3) Received unknown parameters
for command listZones. Unknown parameters : projectid
2015-07-16 12:13:58,786 WARN [c.c.a.d.ParamGenericValidationWorker]
(catalina-exec-19:ctx-ef9b46ae ctx-1ec7c374) Received unknown parameters
for command listZones. Unknown parameters : projectid
2015-07-16 12:13:58,991 WARN [c.c.a.d.ParamGenericValidationWorker]
(catalina-exec-3:ctx-c40727ff ctx-129cff5b) Received unknown parameters for
command listHosts. Unknown parameters : projectid
2015-07-16 12:14:01,369 WARN [c.c.a.d.ParamGenericValidationWorker]
(catalina-exec-9:ctx-d39d30d5 ctx-7d1296f3) Received unknown parameters for
command listHosts. Unknown parameters : projectid
2015-07-16 12:14:01,553 WARN [c.c.a.d.ParamGenericValidationWorker]
(catalina-exec-6:ctx-f02aae15 ctx-b675547d) Received unknown parameters for
command listDedicatedHosts. Unknown parameters : projectid
2015-07-16 12:14:01,725 WARN [c.c.a.d.ParamGenericValidationWorker]
(catalina-exec-25:ctx-4de9059a ctx-15ac62c3) Received unknown parameters
for command listConfigurations. Unknown parameters : projectid
2015-07-16 12:14:01,908 WARN [c.c.a.d.ParamGenericValidationWorker]
(catalina-exec-23:ctx-25fb3589 ctx-d4274e7c) Received unknown parameters
for command listOsCategories. Unknown parameters : projectid
2015-07-16 12:14:02,081 WARN [c.c.a.d.ParamGenericValidationWorker]
(catalina-exec-2:ctx-a90cf3ca ctx-6e7a01fc) Received unknown parameters for
command listHosts. Unknown parameters : projectid
2015-07-16 12:14:02,266 WARN [c.c.a.d.ParamGenericValidationWorker]
(catalina-exec-12:ctx-b8f987fa ctx-8a7f2922) Received unknown parameters
for command listDedicatedHosts. Unknown parameters : projectid
2015-07-16 12:14:02,442 WARN [c.c.a.d.ParamGenericValidationWorker]
(catalina-exec-7:ctx-64be7d4b ctx-36a9864d) Received unknown parameters for
command listConfigurations. Unknown parameters : projectid
2015-07-16 12:14:02,650 WARN [c.c.a.d.ParamGenericValidationWorker]
(catalina-exec-1:ctx-9b0043f1 ctx-36482b14) Received unknown parameters for
command listOsCategories. Unknown parameters : projectid
2015-07-16 12:14:14,981 WARN [c.c.a.d.ParamGenericValidationWorker]
(catalina-exec-16:ctx-5d62538b ctx-31cf1646) Received unknown parameters
for command listZones. Unknown parameters : projectid
2015-07-16 12:14:17,445 WARN [c.c.a.d.ParamGenericValidationWorker]
(catalina-exec-17:ctx-ef67f80f ctx-e953915f) Received unknown parameters
for command listZones. Unknown parameters : projectid
2015-07-16 12:14:17,654 WARN [c.c.a.d.ParamGenericValidationWorker]
(catalina-exec-11:ctx-7a3e6ce3 ctx-7d06a5a4) Received unknown parameters
for command listHosts. Unknown parameters : projectid
2015-07-16 12:14:24,537 WARN [c.c.a.d.ParamGenericValidationWorker]
(catalina-exec-4:ctx-596c9e4b ctx-90642717) Received unknown parameters for
command findHostsForMigration. Unknown parameters : projectid
2015-07-16 12:14:28,684 WARN [c.c.a.d.ParamGenericValidationWorker]
(catalina-exec-20:ctx-2b17bb98 ctx-a489da47) Received unknown parameters
for command migrateVirtualMachine. Unknown parameters : projectid
2015-07-16 12:14:28,708 WARN [c.c.a.d.ParamGenericValidationWorker]
(API-Job-Executor-55:ctx-0b76e8ae job-409 ctx-d346cca9) Received unknown
parameters for command migrateVirtualMachine. Unknown parameters : projectid
2015-07-16 12:14:31,880 WARN [c.c.a.d.ParamGenericValidationWorker]
(catalina-exec-10:ctx-b9b78739 ctx-bc42f26e) Received unknown parameters
for command queryAsyncJobResult. Unknown parameters : projectid
2015-07-16 12:14:34,873 WARN [c.c.a.d.ParamGenericValidationWorker]
(catalina-exec-22:ctx-22941163 ctx-737b322d) Received unknown parameters
for command queryAsyncJobResult. Unknown parameters : projectid
2015-07-16 12:15:47,758 WARN [c.c.a.d.ParamGenericValidationWorker]
(catalina-exec-25:ctx-bd9f0a98 ctx-97c7b112) Received unknown parameters
for command listNetworkOfferings. Unknown parameters : projectid
2015-07-16 12:15:50,466 WARN [c.c.a.d.ParamGenericValidationWorker]
(catalina-exec-16:ctx-c10f59b2 ctx-405d4cf3) Received unknown parameters
for command listZones. Unknown parameters : projectid
2015-07-16 12:15:54,480 WARN [c.c.a.d.ParamGenericValidationWorker]
(catalina-exec-17:ctx-9bb4f185 ctx-9f2b37a9) Received unknown parameters
for command listZones. Unknown parameters : projectid
2015-07-16 12:15:54,684 WARN [c.c.a.d.ParamGenericValidationWorker]
(catalina-exec-11:ctx-6a51a841 ctx-a0d37d54) Received unknown parameters
for command listHosts. Unknown parameters : projectid
2015-07-16 12:15:58,044 WARN [c.c.a.d.ParamGenericValidationWorker]
(catalina-exec-22:ctx-656c8c86 ctx-aef2da97) Received unknown parameters
for command listNetworkOfferings. Unknown parameters : projectid
2015-07-16 12:16:37,381 DEBUG [c.c.c.CapacityManagerImpl]
(API-Job-Executor-57:ctx-e1ed3c62 job-412 ctx-e861a024) STATS: Failed to
alloc resource from host: 3 reservedCpu: 0, requested cpu: 4000,
reservedMem: 0, requested mem: 2147483648
2015-07-16 12:21:18,239 WARN [c.c.a.m.AgentManagerImpl]
(AgentMonitor-1:ctx-9eef326e) Disconnect agent for CPVM/SSVM due to
physical connection close. host: 9
2015-07-16 12:21:18,243 INFO [c.c.u.e.CSExceptionErrorCode]
(StatsCollector-3:ctx-eafd1ed4) Could not find exception:
com.cloud.exception.OperationTimedoutException in error code list for
exceptions
2015-07-16 12:21:18,243 WARN [c.c.a.m.AgentAttache]
(StatsCollector-3:ctx-eafd1ed4) Seq 9-8454382400481264659: Timed out on null
2015-07-16 12:21:18,243 DEBUG [o.a.c.s.RemoteHostEndPoint]
(StatsCollector-3:ctx-eafd1ed4) Failed to send command, due to Agent:9,
com.cloud.exception.OperationTimedoutException: Commands
8454382400481264659 to Host 9 timed out after 3600
2015-07-16 12:21:18,243 ERROR [c.c.s.StatsCollector]
(StatsCollector-3:ctx-eafd1ed4) Error trying to retrieve storage stats
com.cloud.utils.exception.CloudRuntimeException: Failed to send command,
due to Agent:9, com.cloud.exception.OperationTimedoutException: Commands
8454382400481264659 to Host 9 timed out after 3600
2015-07-16 12:22:09,951 WARN [o.a.c.alerts] (HA-1:ctx-6b60e29b)
alertType:: 13 // dataCenterId:: 0 // podId:: 0 // clusterId:: null //
message:: No usage server process running
[root@1q2 ~]# grep -i -E 'warn|exception|unable|fail|error'
/var/log/cloudstack/management/catalina.out|tail -50
WARN [o.a.c.alerts] (HA-1:ctx-830dab68) alertType:: 13 // dataCenterId::
0 // podId:: 0 // clusterId:: null // message:: No usage server process
running
WARN [o.a.c.alerts] (HA-1:ctx-e45484a1) alertType:: 13 // dataCenterId::
0 // podId:: 0 // clusterId:: null // message:: No usage server process
running
WARN [o.a.c.alerts] (HA-1:ctx-c27a224e) alertType:: 13 // dataCenterId::
0 // podId:: 0 // clusterId:: null // message:: No usage server process
running
WARN [o.a.c.alerts] (HA-1:ctx-259fdd28) alertType:: 13 // dataCenterId::
0 // podId:: 0 // clusterId:: null // message:: No usage server process
running
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-16:ctx-40e06aca
ctx-d419dc55) Received unknown parameters for command listNetworkOfferings.
Unknown parameters : projectid
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-22:ctx-4b3562fd
ctx-3341445a) Received unknown parameters for command listZones. Unknown
parameters : projectid
WARN [o.a.c.alerts] (HA-1:ctx-f34fcc40) alertType:: 13 // dataCenterId::
0 // podId:: 0 // clusterId:: null // message:: No usage server process
running
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-19:ctx-8f8e296e
ctx-a6f598c1) Received unknown parameters for command listZones. Unknown
parameters : projectid
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-3:ctx-8825ea69
ctx-fb7312c0) Received unknown parameters for command listZones. Unknown
parameters : projectid
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-9:ctx-d6a4922e
ctx-b3bad36d) Received unknown parameters for command listHosts. Unknown
parameters : projectid
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-17:ctx-f7e28168
ctx-e966ba7a) Received unknown parameters for command listServiceOfferings.
Unknown parameters : listall
WARN [o.a.c.alerts] (HA-1:ctx-42103a07) alertType:: 13 // dataCenterId::
0 // podId:: 0 // clusterId:: null // message:: No usage server process
running
WARN [o.a.c.alerts] (HA-1:ctx-5cac1938) alertType:: 13 // dataCenterId::
0 // podId:: 0 // clusterId:: null // message:: No usage server process
running
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-14:ctx-b1ba1de6
ctx-5c5114f1) Received unknown parameters for command listNetworkOfferings.
Unknown parameters : projectid
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-4:ctx-78cb8329
ctx-ea77f742) Received unknown parameters for command listNetworkOfferings.
Unknown parameters : projectid
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-19:ctx-6e2ab11e
ctx-f0bc3e8d) Received unknown parameters for command
expungeVirtualMachine. Unknown parameters : projectid
WARN [c.c.a.d.ParamGenericValidationWorker]
(API-Job-Executor-54:ctx-a6765444 job-407 ctx-e7d4ae6e) Received unknown
parameters for command expungeVirtualMachine. Unknown parameters : projectid
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-3:ctx-ca71fefa
ctx-3a46d16f) Received unknown parameters for command queryAsyncJobResult.
Unknown parameters : projectid
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-2:ctx-9773fe5c
ctx-20f2c53d) Received unknown parameters for command listNetworkOfferings.
Unknown parameters : projectid
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-4:ctx-986e5695
ctx-cfd9dc2a) Received unknown parameters for command listNetworkOfferings.
Unknown parameters : projectid
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-21:ctx-3ed5162d
ctx-0b378da3) Received unknown parameters for command listZones. Unknown
parameters : projectid
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-19:ctx-ef9b46ae
ctx-1ec7c374) Received unknown parameters for command listZones. Unknown
parameters : projectid
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-3:ctx-c40727ff
ctx-129cff5b) Received unknown parameters for command listHosts. Unknown
parameters : projectid
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-9:ctx-d39d30d5
ctx-7d1296f3) Received unknown parameters for command listHosts. Unknown
parameters : projectid
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-6:ctx-f02aae15
ctx-b675547d) Received unknown parameters for command listDedicatedHosts.
Unknown parameters : projectid
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-25:ctx-4de9059a
ctx-15ac62c3) Received unknown parameters for command listConfigurations.
Unknown parameters : projectid
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-23:ctx-25fb3589
ctx-d4274e7c) Received unknown parameters for command listOsCategories.
Unknown parameters : projectid
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-2:ctx-a90cf3ca
ctx-6e7a01fc) Received unknown parameters for command listHosts. Unknown
parameters : projectid
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-12:ctx-b8f987fa
ctx-8a7f2922) Received unknown parameters for command listDedicatedHosts.
Unknown parameters : projectid
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-7:ctx-64be7d4b
ctx-36a9864d) Received unknown parameters for command listConfigurations.
Unknown parameters : projectid
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-1:ctx-9b0043f1
ctx-36482b14) Received unknown parameters for command listOsCategories.
Unknown parameters : projectid
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-16:ctx-5d62538b
ctx-31cf1646) Received unknown parameters for command listZones. Unknown
parameters : projectid
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-17:ctx-ef67f80f
ctx-e953915f) Received unknown parameters for command listZones. Unknown
parameters : projectid
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-11:ctx-7a3e6ce3
ctx-7d06a5a4) Received unknown parameters for command listHosts. Unknown
parameters : projectid
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-4:ctx-596c9e4b
ctx-90642717) Received unknown parameters for command
findHostsForMigration. Unknown parameters : projectid
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-20:ctx-2b17bb98
ctx-a489da47) Received unknown parameters for command
migrateVirtualMachine. Unknown parameters : projectid
WARN [c.c.a.d.ParamGenericValidationWorker]
(API-Job-Executor-55:ctx-0b76e8ae job-409 ctx-d346cca9) Received unknown
parameters for command migrateVirtualMachine. Unknown parameters : projectid
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-10:ctx-b9b78739
ctx-bc42f26e) Received unknown parameters for command queryAsyncJobResult.
Unknown parameters : projectid
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-22:ctx-22941163
ctx-737b322d) Received unknown parameters for command queryAsyncJobResult.
Unknown parameters : projectid
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-25:ctx-bd9f0a98
ctx-97c7b112) Received unknown parameters for command listNetworkOfferings.
Unknown parameters : projectid
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-16:ctx-c10f59b2
ctx-405d4cf3) Received unknown parameters for command listZones. Unknown
parameters : projectid
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-17:ctx-9bb4f185
ctx-9f2b37a9) Received unknown parameters for command listZones. Unknown
parameters : projectid
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-11:ctx-6a51a841
ctx-a0d37d54) Received unknown parameters for command listHosts. Unknown
parameters : projectid
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-22:ctx-656c8c86
ctx-aef2da97) Received unknown parameters for command listNetworkOfferings.
Unknown parameters : projectid
WARN [c.c.a.m.AgentManagerImpl] (AgentMonitor-1:ctx-9eef326e) Disconnect
agent for CPVM/SSVM due to physical connection close. host: 9
INFO [c.c.u.e.CSExceptionErrorCode] (StatsCollector-3:ctx-eafd1ed4) Could
not find exception: com.cloud.exception.OperationTimedoutException in error
code list for exceptions
WARN [c.c.a.m.AgentAttache] (StatsCollector-3:ctx-eafd1ed4) Seq
9-8454382400481264659: Timed out on null
ERROR [c.c.s.StatsCollector] (StatsCollector-3:ctx-eafd1ed4) Error trying
to retrieve storage stats
com.cloud.utils.exception.CloudRuntimeException: Failed to send command,
due to Agent:9, com.cloud.exception.OperationTimedoutException: Commands
8454382400481264659 to Host 9 timed out after 3600
WARN [o.a.c.alerts] (HA-1:ctx-6b60e29b) alertType:: 13 // dataCenterId::
0 // podId:: 0 // clusterId:: null // message:: No usage server process
running
PS: I was loggen in the vm with offer HA, and I started a ping. I lost the
connection after 30 icmp_seq.
Thanks!
On Wed, Jul 15, 2015 at 7:11 PM, Somesh Naidu <So...@citrix.com>>
wrote:
"TAG H.A checked" - did you mean "Offer HA"?
Also, after you shutdown host A, what was the status of the host reported
in Cloudstack? Also, did you check the management server logs to see what
the investigators reported?
Regards,
Somesh
-----Original Message-----
From: Luciano Castro [mailto:luciano.castro@gmail.com]
Sent: Wednesday, July 15, 2015 2:46 PM
To: users@cloudstack.apache.org<ma...@cloudstack.apache.org>
Subject: Fwd: HA feature - KVM - CloudStack 4.5.1
Hi there,
I´m new in the list, and I´m using 4.5.1 CloudStack version, and I
started making some tests with H.A feature for VMs,
but I didn´t have success. I don´t know if it´s a bug, or something that
I´m doing wrong.
Scenario: I have 1 manager server and 2 KVM hosts ('A' and 'B').
1) I created a "service offering" (computing offering) with the TAG H.A
checked.
2) then, I created a VM with this offering, and the VM is in KVM 'A'.
3) I did a shutdown in the KVM host 'A', and I expected that this VM, after
some minutes will be available in another KVM host ('B')
But it didn´t happened.
Any ideias??
thanks!
--
Luciano Castro
--
Luciano Castro
--
Luciano Castro
Re: HA feature - KVM - CloudStack 4.5.1
Posted by Luciano Castro <lu...@gmail.com>.
No problems Somesh, thanks for your help.
Link of log:
https://dl.dropboxusercontent.com/u/6774061/management-server.log.2015-07-16.gz
Luciano
On Fri, Jul 17, 2015 at 12:00 PM, Somesh Naidu <So...@citrix.com>
wrote:
> How large is the management server logs dated 2015-07-16? I would like to
> review the logs. All the information I need from that incident should be in
> there so I don't need any more testing.
>
> Regards,
> Somesh
>
> -----Original Message-----
> From: Luciano Castro [mailto:luciano.castro@gmail.com]
> Sent: Friday, July 17, 2015 7:58 AM
> To: users@cloudstack.apache.org
> Subject: Re: HA feature - KVM - CloudStack 4.5.1
>
> Hi Somesh!
>
>
>
>
>
>
>
>
>
> [root@1q2 ~]# zgrep -i -E
>
> 'SimpleIvestigator|KVMInvestigator|PingInvestigator|ManagementIPSysVMInvestigator'
> /var/log/cloudstack/management/management-server.log.2015-07-16.gz |tail
> -5000 > /tmp/management.txt
> [root@1q2 ~]# cat /tmp/management.txt
> 2015-07-16 12:30:45,452 DEBUG [o.a.c.s.l.r.ExtensionRegistry] (main:null)
> Registering extension [KVMInvestigator] in [Ha Investigators Registry]
> 2015-07-16 12:30:45,452 DEBUG [o.a.c.s.l.r.RegistryLifecycle] (main:null)
> Registered com.cloud.ha.KVMInvestigator@57ceec9a
> 2015-07-16 12:30:45,927 DEBUG [o.a.c.s.l.r.ExtensionRegistry] (main:null)
> Registering extension [PingInvestigator] in [Ha Investigators Registry]
> 2015-07-16 12:30:45,928 DEBUG [o.a.c.s.l.r.ExtensionRegistry] (main:null)
> Registering extension [ManagementIPSysVMInvestigator] in [Ha Investigators
> Registry]
> 2015-07-16 12:30:53,796 INFO [o.a.c.s.l.r.DumpRegistry] (main:null)
> Registry [Ha Investigators Registry] contains [SimpleInvestigator,
> XenServerInvestigator, KVMInv
>
> I searched this log before, but as I thought that had not nothing
> special.
>
> If you want propose to me another scenario of test, I can do it.
>
> Thanks
>
>
> On Thu, Jul 16, 2015 at 7:27 PM, Somesh Naidu <So...@citrix.com>
> wrote:
>
> > What about other investigators, specifically " KVMInvestigator,
> > PingInvestigator"? They report the VMs as alive=false too?
> >
> > Also, it is recommended that you look at the management-sever.log instead
> > of catalina.out (for one, the latter doesn’t have timestamp).
> >
> > Regards,
> > Somesh
> >
> >
> > -----Original Message-----
> > From: Luciano Castro [mailto:luciano.castro@gmail.com]
> > Sent: Thursday, July 16, 2015 1:14 PM
> > To: users@cloudstack.apache.org
> > Subject: Re: HA feature - KVM - CloudStack 4.5.1
> >
> > Hi Somesh!
> >
> >
> > thanks for help.. I did again ,and I collected new logs:
> >
> > My vm_instance name is i-2-39-VM. There was some routers in KVM host 'A'
> > (this one that I powered off now):
> >
> >
> > [root@1q2 ~]# grep -i -E 'SimpleInvestigator.*false'
> > /var/log/cloudstack/management/catalina.out
> > INFO [c.c.h.HighAvailabilityManagerImpl] (HA-Worker-2:ctx-e2f91c9c
> work-3)
> > SimpleInvestigator found VM[DomainRouter|r-4-VM]to be alive? false
> > INFO [c.c.h.HighAvailabilityManagerImpl] (HA-Worker-1:ctx-729acf4f
> work-7)
> > SimpleInvestigator found VM[User|i-23-33-VM]to be alive? false
> > INFO [c.c.h.HighAvailabilityManagerImpl] (HA-Worker-4:ctx-a66a4941
> work-8)
> > SimpleInvestigator found VM[DomainRouter|r-36-VM]to be alive? false
> > INFO [c.c.h.HighAvailabilityManagerImpl] (HA-Worker-1:ctx-5977245e
> > work-10) SimpleInvestigator found VM[User|i-17-26-VM]to be alive? false
> > INFO [c.c.h.HighAvailabilityManagerImpl] (HA-Worker-1:ctx-c7f39be0
> work-9)
> > SimpleInvestigator found VM[DomainRouter|r-32-VM]to be alive? false
> > INFO [c.c.h.HighAvailabilityManagerImpl] (HA-Worker-3:ctx-ad4f5fda
> > work-10) SimpleInvestigator found VM[DomainRouter|r-46-VM]to be alive?
> > false
> > INFO [c.c.h.HighAvailabilityManagerImpl] (HA-Worker-0:ctx-0257f5af
> > work-11) SimpleInvestigator found VM[User|i-4-52-VM]to be alive? false
> > INFO [c.c.h.HighAvailabilityManagerImpl] (HA-Worker-4:ctx-7ddff382
> > work-12) SimpleInvestigator found VM[DomainRouter|r-32-VM]to be alive?
> > false
> > INFO [c.c.h.HighAvailabilityManagerImpl] (HA-Worker-1:ctx-9f79917e
> > work-13) SimpleInvestigator found VM[User|i-2-39-VM]to be alive? false
> >
> >
> >
> > KVM host 'B' agent log (where the machine would be migrate):
> >
> > 2015-07-16 16:58:56,537 INFO [kvm.resource.LibvirtComputingResource]
> > (agentRequest-Handler-4:null) Live migration of instance i-2-39-VM
> > initiated
> > 2015-07-16 16:58:57,540 INFO [kvm.resource.LibvirtComputingResource]
> > (agentRequest-Handler-4:null) Waiting for migration of i-2-39-VM to
> > complete, waited 1000ms
> > 2015-07-16 16:58:58,541 INFO [kvm.resource.LibvirtComputingResource]
> > (agentRequest-Handler-4:null) Waiting for migration of i-2-39-VM to
> > complete, waited 2000ms
> > 2015-07-16 16:58:59,542 INFO [kvm.resource.LibvirtComputingResource]
> > (agentRequest-Handler-4:null) Waiting for migration of i-2-39-VM to
> > complete, waited 3000ms
> > 2015-07-16 16:59:00,543 INFO [kvm.resource.LibvirtComputingResource]
> > (agentRequest-Handler-4:null) Waiting for migration of i-2-39-VM to
> > complete, waited 4000ms
> > 2015-07-16 16:59:01,245 INFO [kvm.resource.LibvirtComputingResource]
> > (agentRequest-Handler-4:null) Migration thread for i-2-39-VM is done
> >
> > It said done for my i-2-39-VM instance, but I can´t ping this host.
> >
> > Luciano
> >
>
>
>
> --
> Luciano Castro
>
--
Luciano Castro
RE: HA feature - KVM - CloudStack 4.5.1
Posted by Somesh Naidu <So...@citrix.com>.
How large is the management server logs dated 2015-07-16? I would like to review the logs. All the information I need from that incident should be in there so I don't need any more testing.
Regards,
Somesh
-----Original Message-----
From: Luciano Castro [mailto:luciano.castro@gmail.com]
Sent: Friday, July 17, 2015 7:58 AM
To: users@cloudstack.apache.org
Subject: Re: HA feature - KVM - CloudStack 4.5.1
Hi Somesh!
[root@1q2 ~]# zgrep -i -E
'SimpleIvestigator|KVMInvestigator|PingInvestigator|ManagementIPSysVMInvestigator'
/var/log/cloudstack/management/management-server.log.2015-07-16.gz |tail
-5000 > /tmp/management.txt
[root@1q2 ~]# cat /tmp/management.txt
2015-07-16 12:30:45,452 DEBUG [o.a.c.s.l.r.ExtensionRegistry] (main:null)
Registering extension [KVMInvestigator] in [Ha Investigators Registry]
2015-07-16 12:30:45,452 DEBUG [o.a.c.s.l.r.RegistryLifecycle] (main:null)
Registered com.cloud.ha.KVMInvestigator@57ceec9a
2015-07-16 12:30:45,927 DEBUG [o.a.c.s.l.r.ExtensionRegistry] (main:null)
Registering extension [PingInvestigator] in [Ha Investigators Registry]
2015-07-16 12:30:45,928 DEBUG [o.a.c.s.l.r.ExtensionRegistry] (main:null)
Registering extension [ManagementIPSysVMInvestigator] in [Ha Investigators
Registry]
2015-07-16 12:30:53,796 INFO [o.a.c.s.l.r.DumpRegistry] (main:null)
Registry [Ha Investigators Registry] contains [SimpleInvestigator,
XenServerInvestigator, KVMInv
I searched this log before, but as I thought that had not nothing special.
If you want propose to me another scenario of test, I can do it.
Thanks
On Thu, Jul 16, 2015 at 7:27 PM, Somesh Naidu <So...@citrix.com>
wrote:
> What about other investigators, specifically " KVMInvestigator,
> PingInvestigator"? They report the VMs as alive=false too?
>
> Also, it is recommended that you look at the management-sever.log instead
> of catalina.out (for one, the latter doesn’t have timestamp).
>
> Regards,
> Somesh
>
>
> -----Original Message-----
> From: Luciano Castro [mailto:luciano.castro@gmail.com]
> Sent: Thursday, July 16, 2015 1:14 PM
> To: users@cloudstack.apache.org
> Subject: Re: HA feature - KVM - CloudStack 4.5.1
>
> Hi Somesh!
>
>
> thanks for help.. I did again ,and I collected new logs:
>
> My vm_instance name is i-2-39-VM. There was some routers in KVM host 'A'
> (this one that I powered off now):
>
>
> [root@1q2 ~]# grep -i -E 'SimpleInvestigator.*false'
> /var/log/cloudstack/management/catalina.out
> INFO [c.c.h.HighAvailabilityManagerImpl] (HA-Worker-2:ctx-e2f91c9c work-3)
> SimpleInvestigator found VM[DomainRouter|r-4-VM]to be alive? false
> INFO [c.c.h.HighAvailabilityManagerImpl] (HA-Worker-1:ctx-729acf4f work-7)
> SimpleInvestigator found VM[User|i-23-33-VM]to be alive? false
> INFO [c.c.h.HighAvailabilityManagerImpl] (HA-Worker-4:ctx-a66a4941 work-8)
> SimpleInvestigator found VM[DomainRouter|r-36-VM]to be alive? false
> INFO [c.c.h.HighAvailabilityManagerImpl] (HA-Worker-1:ctx-5977245e
> work-10) SimpleInvestigator found VM[User|i-17-26-VM]to be alive? false
> INFO [c.c.h.HighAvailabilityManagerImpl] (HA-Worker-1:ctx-c7f39be0 work-9)
> SimpleInvestigator found VM[DomainRouter|r-32-VM]to be alive? false
> INFO [c.c.h.HighAvailabilityManagerImpl] (HA-Worker-3:ctx-ad4f5fda
> work-10) SimpleInvestigator found VM[DomainRouter|r-46-VM]to be alive?
> false
> INFO [c.c.h.HighAvailabilityManagerImpl] (HA-Worker-0:ctx-0257f5af
> work-11) SimpleInvestigator found VM[User|i-4-52-VM]to be alive? false
> INFO [c.c.h.HighAvailabilityManagerImpl] (HA-Worker-4:ctx-7ddff382
> work-12) SimpleInvestigator found VM[DomainRouter|r-32-VM]to be alive?
> false
> INFO [c.c.h.HighAvailabilityManagerImpl] (HA-Worker-1:ctx-9f79917e
> work-13) SimpleInvestigator found VM[User|i-2-39-VM]to be alive? false
>
>
>
> KVM host 'B' agent log (where the machine would be migrate):
>
> 2015-07-16 16:58:56,537 INFO [kvm.resource.LibvirtComputingResource]
> (agentRequest-Handler-4:null) Live migration of instance i-2-39-VM
> initiated
> 2015-07-16 16:58:57,540 INFO [kvm.resource.LibvirtComputingResource]
> (agentRequest-Handler-4:null) Waiting for migration of i-2-39-VM to
> complete, waited 1000ms
> 2015-07-16 16:58:58,541 INFO [kvm.resource.LibvirtComputingResource]
> (agentRequest-Handler-4:null) Waiting for migration of i-2-39-VM to
> complete, waited 2000ms
> 2015-07-16 16:58:59,542 INFO [kvm.resource.LibvirtComputingResource]
> (agentRequest-Handler-4:null) Waiting for migration of i-2-39-VM to
> complete, waited 3000ms
> 2015-07-16 16:59:00,543 INFO [kvm.resource.LibvirtComputingResource]
> (agentRequest-Handler-4:null) Waiting for migration of i-2-39-VM to
> complete, waited 4000ms
> 2015-07-16 16:59:01,245 INFO [kvm.resource.LibvirtComputingResource]
> (agentRequest-Handler-4:null) Migration thread for i-2-39-VM is done
>
> It said done for my i-2-39-VM instance, but I can´t ping this host.
>
> Luciano
>
--
Luciano Castro
Re: HA feature - KVM - CloudStack 4.5.1
Posted by Luciano Castro <lu...@gmail.com>.
Hi Somesh!
[root@1q2 ~]# zgrep -i -E
'SimpleIvestigator|KVMInvestigator|PingInvestigator|ManagementIPSysVMInvestigator'
/var/log/cloudstack/management/management-server.log.2015-07-16.gz |tail
-5000 > /tmp/management.txt
[root@1q2 ~]# cat /tmp/management.txt
2015-07-16 12:30:45,452 DEBUG [o.a.c.s.l.r.ExtensionRegistry] (main:null)
Registering extension [KVMInvestigator] in [Ha Investigators Registry]
2015-07-16 12:30:45,452 DEBUG [o.a.c.s.l.r.RegistryLifecycle] (main:null)
Registered com.cloud.ha.KVMInvestigator@57ceec9a
2015-07-16 12:30:45,927 DEBUG [o.a.c.s.l.r.ExtensionRegistry] (main:null)
Registering extension [PingInvestigator] in [Ha Investigators Registry]
2015-07-16 12:30:45,928 DEBUG [o.a.c.s.l.r.ExtensionRegistry] (main:null)
Registering extension [ManagementIPSysVMInvestigator] in [Ha Investigators
Registry]
2015-07-16 12:30:53,796 INFO [o.a.c.s.l.r.DumpRegistry] (main:null)
Registry [Ha Investigators Registry] contains [SimpleInvestigator,
XenServerInvestigator, KVMInv
I searched this log before, but as I thought that had not nothing special.
If you want propose to me another scenario of test, I can do it.
Thanks
On Thu, Jul 16, 2015 at 7:27 PM, Somesh Naidu <So...@citrix.com>
wrote:
> What about other investigators, specifically " KVMInvestigator,
> PingInvestigator"? They report the VMs as alive=false too?
>
> Also, it is recommended that you look at the management-sever.log instead
> of catalina.out (for one, the latter doesn’t have timestamp).
>
> Regards,
> Somesh
>
>
> -----Original Message-----
> From: Luciano Castro [mailto:luciano.castro@gmail.com]
> Sent: Thursday, July 16, 2015 1:14 PM
> To: users@cloudstack.apache.org
> Subject: Re: HA feature - KVM - CloudStack 4.5.1
>
> Hi Somesh!
>
>
> thanks for help.. I did again ,and I collected new logs:
>
> My vm_instance name is i-2-39-VM. There was some routers in KVM host 'A'
> (this one that I powered off now):
>
>
> [root@1q2 ~]# grep -i -E 'SimpleInvestigator.*false'
> /var/log/cloudstack/management/catalina.out
> INFO [c.c.h.HighAvailabilityManagerImpl] (HA-Worker-2:ctx-e2f91c9c work-3)
> SimpleInvestigator found VM[DomainRouter|r-4-VM]to be alive? false
> INFO [c.c.h.HighAvailabilityManagerImpl] (HA-Worker-1:ctx-729acf4f work-7)
> SimpleInvestigator found VM[User|i-23-33-VM]to be alive? false
> INFO [c.c.h.HighAvailabilityManagerImpl] (HA-Worker-4:ctx-a66a4941 work-8)
> SimpleInvestigator found VM[DomainRouter|r-36-VM]to be alive? false
> INFO [c.c.h.HighAvailabilityManagerImpl] (HA-Worker-1:ctx-5977245e
> work-10) SimpleInvestigator found VM[User|i-17-26-VM]to be alive? false
> INFO [c.c.h.HighAvailabilityManagerImpl] (HA-Worker-1:ctx-c7f39be0 work-9)
> SimpleInvestigator found VM[DomainRouter|r-32-VM]to be alive? false
> INFO [c.c.h.HighAvailabilityManagerImpl] (HA-Worker-3:ctx-ad4f5fda
> work-10) SimpleInvestigator found VM[DomainRouter|r-46-VM]to be alive?
> false
> INFO [c.c.h.HighAvailabilityManagerImpl] (HA-Worker-0:ctx-0257f5af
> work-11) SimpleInvestigator found VM[User|i-4-52-VM]to be alive? false
> INFO [c.c.h.HighAvailabilityManagerImpl] (HA-Worker-4:ctx-7ddff382
> work-12) SimpleInvestigator found VM[DomainRouter|r-32-VM]to be alive?
> false
> INFO [c.c.h.HighAvailabilityManagerImpl] (HA-Worker-1:ctx-9f79917e
> work-13) SimpleInvestigator found VM[User|i-2-39-VM]to be alive? false
>
>
>
> KVM host 'B' agent log (where the machine would be migrate):
>
> 2015-07-16 16:58:56,537 INFO [kvm.resource.LibvirtComputingResource]
> (agentRequest-Handler-4:null) Live migration of instance i-2-39-VM
> initiated
> 2015-07-16 16:58:57,540 INFO [kvm.resource.LibvirtComputingResource]
> (agentRequest-Handler-4:null) Waiting for migration of i-2-39-VM to
> complete, waited 1000ms
> 2015-07-16 16:58:58,541 INFO [kvm.resource.LibvirtComputingResource]
> (agentRequest-Handler-4:null) Waiting for migration of i-2-39-VM to
> complete, waited 2000ms
> 2015-07-16 16:58:59,542 INFO [kvm.resource.LibvirtComputingResource]
> (agentRequest-Handler-4:null) Waiting for migration of i-2-39-VM to
> complete, waited 3000ms
> 2015-07-16 16:59:00,543 INFO [kvm.resource.LibvirtComputingResource]
> (agentRequest-Handler-4:null) Waiting for migration of i-2-39-VM to
> complete, waited 4000ms
> 2015-07-16 16:59:01,245 INFO [kvm.resource.LibvirtComputingResource]
> (agentRequest-Handler-4:null) Migration thread for i-2-39-VM is done
>
> It said done for my i-2-39-VM instance, but I can´t ping this host.
>
> Luciano
>
--
Luciano Castro
RE: HA feature - KVM - CloudStack 4.5.1
Posted by Somesh Naidu <So...@citrix.com>.
What about other investigators, specifically " KVMInvestigator, PingInvestigator"? They report the VMs as alive=false too?
Also, it is recommended that you look at the management-sever.log instead of catalina.out (for one, the latter doesn’t have timestamp).
Regards,
Somesh
-----Original Message-----
From: Luciano Castro [mailto:luciano.castro@gmail.com]
Sent: Thursday, July 16, 2015 1:14 PM
To: users@cloudstack.apache.org
Subject: Re: HA feature - KVM - CloudStack 4.5.1
Hi Somesh!
thanks for help.. I did again ,and I collected new logs:
My vm_instance name is i-2-39-VM. There was some routers in KVM host 'A'
(this one that I powered off now):
[root@1q2 ~]# grep -i -E 'SimpleInvestigator.*false'
/var/log/cloudstack/management/catalina.out
INFO [c.c.h.HighAvailabilityManagerImpl] (HA-Worker-2:ctx-e2f91c9c work-3)
SimpleInvestigator found VM[DomainRouter|r-4-VM]to be alive? false
INFO [c.c.h.HighAvailabilityManagerImpl] (HA-Worker-1:ctx-729acf4f work-7)
SimpleInvestigator found VM[User|i-23-33-VM]to be alive? false
INFO [c.c.h.HighAvailabilityManagerImpl] (HA-Worker-4:ctx-a66a4941 work-8)
SimpleInvestigator found VM[DomainRouter|r-36-VM]to be alive? false
INFO [c.c.h.HighAvailabilityManagerImpl] (HA-Worker-1:ctx-5977245e
work-10) SimpleInvestigator found VM[User|i-17-26-VM]to be alive? false
INFO [c.c.h.HighAvailabilityManagerImpl] (HA-Worker-1:ctx-c7f39be0 work-9)
SimpleInvestigator found VM[DomainRouter|r-32-VM]to be alive? false
INFO [c.c.h.HighAvailabilityManagerImpl] (HA-Worker-3:ctx-ad4f5fda
work-10) SimpleInvestigator found VM[DomainRouter|r-46-VM]to be alive? false
INFO [c.c.h.HighAvailabilityManagerImpl] (HA-Worker-0:ctx-0257f5af
work-11) SimpleInvestigator found VM[User|i-4-52-VM]to be alive? false
INFO [c.c.h.HighAvailabilityManagerImpl] (HA-Worker-4:ctx-7ddff382
work-12) SimpleInvestigator found VM[DomainRouter|r-32-VM]to be alive? false
INFO [c.c.h.HighAvailabilityManagerImpl] (HA-Worker-1:ctx-9f79917e
work-13) SimpleInvestigator found VM[User|i-2-39-VM]to be alive? false
KVM host 'B' agent log (where the machine would be migrate):
2015-07-16 16:58:56,537 INFO [kvm.resource.LibvirtComputingResource]
(agentRequest-Handler-4:null) Live migration of instance i-2-39-VM initiated
2015-07-16 16:58:57,540 INFO [kvm.resource.LibvirtComputingResource]
(agentRequest-Handler-4:null) Waiting for migration of i-2-39-VM to
complete, waited 1000ms
2015-07-16 16:58:58,541 INFO [kvm.resource.LibvirtComputingResource]
(agentRequest-Handler-4:null) Waiting for migration of i-2-39-VM to
complete, waited 2000ms
2015-07-16 16:58:59,542 INFO [kvm.resource.LibvirtComputingResource]
(agentRequest-Handler-4:null) Waiting for migration of i-2-39-VM to
complete, waited 3000ms
2015-07-16 16:59:00,543 INFO [kvm.resource.LibvirtComputingResource]
(agentRequest-Handler-4:null) Waiting for migration of i-2-39-VM to
complete, waited 4000ms
2015-07-16 16:59:01,245 INFO [kvm.resource.LibvirtComputingResource]
(agentRequest-Handler-4:null) Migration thread for i-2-39-VM is done
It said done for my i-2-39-VM instance, but I can´t ping this host.
Luciano
Re: HA feature - KVM - CloudStack 4.5.1
Posted by Luciano Castro <lu...@gmail.com>.
Hi Somesh!
thanks for help.. I did again ,and I collected new logs:
My vm_instance name is i-2-39-VM. There was some routers in KVM host 'A'
(this one that I powered off now):
[root@1q2 ~]# grep -i -E 'SimpleInvestigator.*false'
/var/log/cloudstack/management/catalina.out
INFO [c.c.h.HighAvailabilityManagerImpl] (HA-Worker-2:ctx-e2f91c9c work-3)
SimpleInvestigator found VM[DomainRouter|r-4-VM]to be alive? false
INFO [c.c.h.HighAvailabilityManagerImpl] (HA-Worker-1:ctx-729acf4f work-7)
SimpleInvestigator found VM[User|i-23-33-VM]to be alive? false
INFO [c.c.h.HighAvailabilityManagerImpl] (HA-Worker-4:ctx-a66a4941 work-8)
SimpleInvestigator found VM[DomainRouter|r-36-VM]to be alive? false
INFO [c.c.h.HighAvailabilityManagerImpl] (HA-Worker-1:ctx-5977245e
work-10) SimpleInvestigator found VM[User|i-17-26-VM]to be alive? false
INFO [c.c.h.HighAvailabilityManagerImpl] (HA-Worker-1:ctx-c7f39be0 work-9)
SimpleInvestigator found VM[DomainRouter|r-32-VM]to be alive? false
INFO [c.c.h.HighAvailabilityManagerImpl] (HA-Worker-3:ctx-ad4f5fda
work-10) SimpleInvestigator found VM[DomainRouter|r-46-VM]to be alive? false
INFO [c.c.h.HighAvailabilityManagerImpl] (HA-Worker-0:ctx-0257f5af
work-11) SimpleInvestigator found VM[User|i-4-52-VM]to be alive? false
INFO [c.c.h.HighAvailabilityManagerImpl] (HA-Worker-4:ctx-7ddff382
work-12) SimpleInvestigator found VM[DomainRouter|r-32-VM]to be alive? false
INFO [c.c.h.HighAvailabilityManagerImpl] (HA-Worker-1:ctx-9f79917e
work-13) SimpleInvestigator found VM[User|i-2-39-VM]to be alive? false
KVM host 'B' agent log (where the machine would be migrate):
2015-07-16 16:58:56,537 INFO [kvm.resource.LibvirtComputingResource]
(agentRequest-Handler-4:null) Live migration of instance i-2-39-VM initiated
2015-07-16 16:58:57,540 INFO [kvm.resource.LibvirtComputingResource]
(agentRequest-Handler-4:null) Waiting for migration of i-2-39-VM to
complete, waited 1000ms
2015-07-16 16:58:58,541 INFO [kvm.resource.LibvirtComputingResource]
(agentRequest-Handler-4:null) Waiting for migration of i-2-39-VM to
complete, waited 2000ms
2015-07-16 16:58:59,542 INFO [kvm.resource.LibvirtComputingResource]
(agentRequest-Handler-4:null) Waiting for migration of i-2-39-VM to
complete, waited 3000ms
2015-07-16 16:59:00,543 INFO [kvm.resource.LibvirtComputingResource]
(agentRequest-Handler-4:null) Waiting for migration of i-2-39-VM to
complete, waited 4000ms
2015-07-16 16:59:01,245 INFO [kvm.resource.LibvirtComputingResource]
(agentRequest-Handler-4:null) Migration thread for i-2-39-VM is done
It said done for my i-2-39-VM instance, but I can´t ping this host.
Luciano
RE: HA feature - KVM - CloudStack 4.5.1
Posted by Somesh Naidu <So...@citrix.com>.
Ok.
I don’t think searching for errors/failures would help. This may not be an error situation from cloudstack perspective.
We need to search for these - [SimpleInvestigator, XenServerInvestigator, KVMInvestigator, PingInvestigator, ManagementIPSysVMInvestigator, HypervInvestigator, VMwareInvestigator]. These investigators are responsible for determining the state of the VM at host disconnect based on which decision to HA is made.
Regards,
Somesh
-----Original Message-----
From: Luciano Castro [mailto:luciano.castro@gmail.com]
Sent: Thursday, July 16, 2015 8:34 AM
To: users@cloudstack.apache.org
Subject: Re: HA feature - KVM - CloudStack 4.5.1
Hi!
* Yes, Offer HA.
* The status os KVM hosts reported is "Disconnected", and curiously the VM
status is still "Running..."
I did the same test again:
1) shutdown KVM host 12:13 UTC
2) Logs (management server)
[root@1q2 ~]# grep -i -E 'warn|exception|unable|fail|error'
/var/log/cloudstack/management/management-server.log|tail -50
2015-07-16 12:13:21,195 WARN [c.c.a.d.ParamGenericValidationWorker]
(catalina-exec-14:ctx-b1ba1de6 ctx-5c5114f1) Received unknown parameters
for command listNetworkOfferings. Unknown parameters : projectid
2015-07-16 12:13:31,395 WARN [c.c.a.d.ParamGenericValidationWorker]
(catalina-exec-4:ctx-78cb8329 ctx-ea77f742) Received unknown parameters for
command listNetworkOfferings. Unknown parameters : projectid
2015-07-16 12:13:40,975 WARN [c.c.a.d.ParamGenericValidationWorker]
(catalina-exec-19:ctx-6e2ab11e ctx-f0bc3e8d) Received unknown parameters
for command expungeVirtualMachine. Unknown parameters : projectid
2015-07-16 12:13:40,998 WARN [c.c.a.d.ParamGenericValidationWorker]
(API-Job-Executor-54:ctx-a6765444 job-407 ctx-e7d4ae6e) Received unknown
parameters for command expungeVirtualMachine. Unknown parameters : projectid
2015-07-16 12:13:44,162 WARN [c.c.a.d.ParamGenericValidationWorker]
(catalina-exec-3:ctx-ca71fefa ctx-3a46d16f) Received unknown parameters for
command queryAsyncJobResult. Unknown parameters : projectid
2015-07-16 12:13:47,163 WARN [c.c.a.d.ParamGenericValidationWorker]
(catalina-exec-2:ctx-9773fe5c ctx-20f2c53d) Received unknown parameters for
command listNetworkOfferings. Unknown parameters : projectid
2015-07-16 12:13:51,478 WARN [c.c.a.d.ParamGenericValidationWorker]
(catalina-exec-4:ctx-986e5695 ctx-cfd9dc2a) Received unknown parameters for
command listNetworkOfferings. Unknown parameters : projectid
2015-07-16 12:13:57,139 WARN [c.c.a.d.ParamGenericValidationWorker]
(catalina-exec-21:ctx-3ed5162d ctx-0b378da3) Received unknown parameters
for command listZones. Unknown parameters : projectid
2015-07-16 12:13:58,786 WARN [c.c.a.d.ParamGenericValidationWorker]
(catalina-exec-19:ctx-ef9b46ae ctx-1ec7c374) Received unknown parameters
for command listZones. Unknown parameters : projectid
2015-07-16 12:13:58,991 WARN [c.c.a.d.ParamGenericValidationWorker]
(catalina-exec-3:ctx-c40727ff ctx-129cff5b) Received unknown parameters for
command listHosts. Unknown parameters : projectid
2015-07-16 12:14:01,369 WARN [c.c.a.d.ParamGenericValidationWorker]
(catalina-exec-9:ctx-d39d30d5 ctx-7d1296f3) Received unknown parameters for
command listHosts. Unknown parameters : projectid
2015-07-16 12:14:01,553 WARN [c.c.a.d.ParamGenericValidationWorker]
(catalina-exec-6:ctx-f02aae15 ctx-b675547d) Received unknown parameters for
command listDedicatedHosts. Unknown parameters : projectid
2015-07-16 12:14:01,725 WARN [c.c.a.d.ParamGenericValidationWorker]
(catalina-exec-25:ctx-4de9059a ctx-15ac62c3) Received unknown parameters
for command listConfigurations. Unknown parameters : projectid
2015-07-16 12:14:01,908 WARN [c.c.a.d.ParamGenericValidationWorker]
(catalina-exec-23:ctx-25fb3589 ctx-d4274e7c) Received unknown parameters
for command listOsCategories. Unknown parameters : projectid
2015-07-16 12:14:02,081 WARN [c.c.a.d.ParamGenericValidationWorker]
(catalina-exec-2:ctx-a90cf3ca ctx-6e7a01fc) Received unknown parameters for
command listHosts. Unknown parameters : projectid
2015-07-16 12:14:02,266 WARN [c.c.a.d.ParamGenericValidationWorker]
(catalina-exec-12:ctx-b8f987fa ctx-8a7f2922) Received unknown parameters
for command listDedicatedHosts. Unknown parameters : projectid
2015-07-16 12:14:02,442 WARN [c.c.a.d.ParamGenericValidationWorker]
(catalina-exec-7:ctx-64be7d4b ctx-36a9864d) Received unknown parameters for
command listConfigurations. Unknown parameters : projectid
2015-07-16 12:14:02,650 WARN [c.c.a.d.ParamGenericValidationWorker]
(catalina-exec-1:ctx-9b0043f1 ctx-36482b14) Received unknown parameters for
command listOsCategories. Unknown parameters : projectid
2015-07-16 12:14:14,981 WARN [c.c.a.d.ParamGenericValidationWorker]
(catalina-exec-16:ctx-5d62538b ctx-31cf1646) Received unknown parameters
for command listZones. Unknown parameters : projectid
2015-07-16 12:14:17,445 WARN [c.c.a.d.ParamGenericValidationWorker]
(catalina-exec-17:ctx-ef67f80f ctx-e953915f) Received unknown parameters
for command listZones. Unknown parameters : projectid
2015-07-16 12:14:17,654 WARN [c.c.a.d.ParamGenericValidationWorker]
(catalina-exec-11:ctx-7a3e6ce3 ctx-7d06a5a4) Received unknown parameters
for command listHosts. Unknown parameters : projectid
2015-07-16 12:14:24,537 WARN [c.c.a.d.ParamGenericValidationWorker]
(catalina-exec-4:ctx-596c9e4b ctx-90642717) Received unknown parameters for
command findHostsForMigration. Unknown parameters : projectid
2015-07-16 12:14:28,684 WARN [c.c.a.d.ParamGenericValidationWorker]
(catalina-exec-20:ctx-2b17bb98 ctx-a489da47) Received unknown parameters
for command migrateVirtualMachine. Unknown parameters : projectid
2015-07-16 12:14:28,708 WARN [c.c.a.d.ParamGenericValidationWorker]
(API-Job-Executor-55:ctx-0b76e8ae job-409 ctx-d346cca9) Received unknown
parameters for command migrateVirtualMachine. Unknown parameters : projectid
2015-07-16 12:14:31,880 WARN [c.c.a.d.ParamGenericValidationWorker]
(catalina-exec-10:ctx-b9b78739 ctx-bc42f26e) Received unknown parameters
for command queryAsyncJobResult. Unknown parameters : projectid
2015-07-16 12:14:34,873 WARN [c.c.a.d.ParamGenericValidationWorker]
(catalina-exec-22:ctx-22941163 ctx-737b322d) Received unknown parameters
for command queryAsyncJobResult. Unknown parameters : projectid
2015-07-16 12:15:47,758 WARN [c.c.a.d.ParamGenericValidationWorker]
(catalina-exec-25:ctx-bd9f0a98 ctx-97c7b112) Received unknown parameters
for command listNetworkOfferings. Unknown parameters : projectid
2015-07-16 12:15:50,466 WARN [c.c.a.d.ParamGenericValidationWorker]
(catalina-exec-16:ctx-c10f59b2 ctx-405d4cf3) Received unknown parameters
for command listZones. Unknown parameters : projectid
2015-07-16 12:15:54,480 WARN [c.c.a.d.ParamGenericValidationWorker]
(catalina-exec-17:ctx-9bb4f185 ctx-9f2b37a9) Received unknown parameters
for command listZones. Unknown parameters : projectid
2015-07-16 12:15:54,684 WARN [c.c.a.d.ParamGenericValidationWorker]
(catalina-exec-11:ctx-6a51a841 ctx-a0d37d54) Received unknown parameters
for command listHosts. Unknown parameters : projectid
2015-07-16 12:15:58,044 WARN [c.c.a.d.ParamGenericValidationWorker]
(catalina-exec-22:ctx-656c8c86 ctx-aef2da97) Received unknown parameters
for command listNetworkOfferings. Unknown parameters : projectid
2015-07-16 12:16:37,381 DEBUG [c.c.c.CapacityManagerImpl]
(API-Job-Executor-57:ctx-e1ed3c62 job-412 ctx-e861a024) STATS: Failed to
alloc resource from host: 3 reservedCpu: 0, requested cpu: 4000,
reservedMem: 0, requested mem: 2147483648
2015-07-16 12:21:18,239 WARN [c.c.a.m.AgentManagerImpl]
(AgentMonitor-1:ctx-9eef326e) Disconnect agent for CPVM/SSVM due to
physical connection close. host: 9
2015-07-16 12:21:18,243 INFO [c.c.u.e.CSExceptionErrorCode]
(StatsCollector-3:ctx-eafd1ed4) Could not find exception:
com.cloud.exception.OperationTimedoutException in error code list for
exceptions
2015-07-16 12:21:18,243 WARN [c.c.a.m.AgentAttache]
(StatsCollector-3:ctx-eafd1ed4) Seq 9-8454382400481264659: Timed out on null
2015-07-16 12:21:18,243 DEBUG [o.a.c.s.RemoteHostEndPoint]
(StatsCollector-3:ctx-eafd1ed4) Failed to send command, due to Agent:9,
com.cloud.exception.OperationTimedoutException: Commands
8454382400481264659 to Host 9 timed out after 3600
2015-07-16 12:21:18,243 ERROR [c.c.s.StatsCollector]
(StatsCollector-3:ctx-eafd1ed4) Error trying to retrieve storage stats
com.cloud.utils.exception.CloudRuntimeException: Failed to send command,
due to Agent:9, com.cloud.exception.OperationTimedoutException: Commands
8454382400481264659 to Host 9 timed out after 3600
2015-07-16 12:22:09,951 WARN [o.a.c.alerts] (HA-1:ctx-6b60e29b)
alertType:: 13 // dataCenterId:: 0 // podId:: 0 // clusterId:: null //
message:: No usage server process running
[root@1q2 ~]# grep -i -E 'warn|exception|unable|fail|error'
/var/log/cloudstack/management/catalina.out|tail -50
WARN [o.a.c.alerts] (HA-1:ctx-830dab68) alertType:: 13 // dataCenterId::
0 // podId:: 0 // clusterId:: null // message:: No usage server process
running
WARN [o.a.c.alerts] (HA-1:ctx-e45484a1) alertType:: 13 // dataCenterId::
0 // podId:: 0 // clusterId:: null // message:: No usage server process
running
WARN [o.a.c.alerts] (HA-1:ctx-c27a224e) alertType:: 13 // dataCenterId::
0 // podId:: 0 // clusterId:: null // message:: No usage server process
running
WARN [o.a.c.alerts] (HA-1:ctx-259fdd28) alertType:: 13 // dataCenterId::
0 // podId:: 0 // clusterId:: null // message:: No usage server process
running
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-16:ctx-40e06aca
ctx-d419dc55) Received unknown parameters for command listNetworkOfferings.
Unknown parameters : projectid
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-22:ctx-4b3562fd
ctx-3341445a) Received unknown parameters for command listZones. Unknown
parameters : projectid
WARN [o.a.c.alerts] (HA-1:ctx-f34fcc40) alertType:: 13 // dataCenterId::
0 // podId:: 0 // clusterId:: null // message:: No usage server process
running
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-19:ctx-8f8e296e
ctx-a6f598c1) Received unknown parameters for command listZones. Unknown
parameters : projectid
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-3:ctx-8825ea69
ctx-fb7312c0) Received unknown parameters for command listZones. Unknown
parameters : projectid
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-9:ctx-d6a4922e
ctx-b3bad36d) Received unknown parameters for command listHosts. Unknown
parameters : projectid
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-17:ctx-f7e28168
ctx-e966ba7a) Received unknown parameters for command listServiceOfferings.
Unknown parameters : listall
WARN [o.a.c.alerts] (HA-1:ctx-42103a07) alertType:: 13 // dataCenterId::
0 // podId:: 0 // clusterId:: null // message:: No usage server process
running
WARN [o.a.c.alerts] (HA-1:ctx-5cac1938) alertType:: 13 // dataCenterId::
0 // podId:: 0 // clusterId:: null // message:: No usage server process
running
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-14:ctx-b1ba1de6
ctx-5c5114f1) Received unknown parameters for command listNetworkOfferings.
Unknown parameters : projectid
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-4:ctx-78cb8329
ctx-ea77f742) Received unknown parameters for command listNetworkOfferings.
Unknown parameters : projectid
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-19:ctx-6e2ab11e
ctx-f0bc3e8d) Received unknown parameters for command
expungeVirtualMachine. Unknown parameters : projectid
WARN [c.c.a.d.ParamGenericValidationWorker]
(API-Job-Executor-54:ctx-a6765444 job-407 ctx-e7d4ae6e) Received unknown
parameters for command expungeVirtualMachine. Unknown parameters : projectid
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-3:ctx-ca71fefa
ctx-3a46d16f) Received unknown parameters for command queryAsyncJobResult.
Unknown parameters : projectid
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-2:ctx-9773fe5c
ctx-20f2c53d) Received unknown parameters for command listNetworkOfferings.
Unknown parameters : projectid
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-4:ctx-986e5695
ctx-cfd9dc2a) Received unknown parameters for command listNetworkOfferings.
Unknown parameters : projectid
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-21:ctx-3ed5162d
ctx-0b378da3) Received unknown parameters for command listZones. Unknown
parameters : projectid
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-19:ctx-ef9b46ae
ctx-1ec7c374) Received unknown parameters for command listZones. Unknown
parameters : projectid
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-3:ctx-c40727ff
ctx-129cff5b) Received unknown parameters for command listHosts. Unknown
parameters : projectid
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-9:ctx-d39d30d5
ctx-7d1296f3) Received unknown parameters for command listHosts. Unknown
parameters : projectid
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-6:ctx-f02aae15
ctx-b675547d) Received unknown parameters for command listDedicatedHosts.
Unknown parameters : projectid
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-25:ctx-4de9059a
ctx-15ac62c3) Received unknown parameters for command listConfigurations.
Unknown parameters : projectid
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-23:ctx-25fb3589
ctx-d4274e7c) Received unknown parameters for command listOsCategories.
Unknown parameters : projectid
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-2:ctx-a90cf3ca
ctx-6e7a01fc) Received unknown parameters for command listHosts. Unknown
parameters : projectid
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-12:ctx-b8f987fa
ctx-8a7f2922) Received unknown parameters for command listDedicatedHosts.
Unknown parameters : projectid
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-7:ctx-64be7d4b
ctx-36a9864d) Received unknown parameters for command listConfigurations.
Unknown parameters : projectid
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-1:ctx-9b0043f1
ctx-36482b14) Received unknown parameters for command listOsCategories.
Unknown parameters : projectid
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-16:ctx-5d62538b
ctx-31cf1646) Received unknown parameters for command listZones. Unknown
parameters : projectid
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-17:ctx-ef67f80f
ctx-e953915f) Received unknown parameters for command listZones. Unknown
parameters : projectid
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-11:ctx-7a3e6ce3
ctx-7d06a5a4) Received unknown parameters for command listHosts. Unknown
parameters : projectid
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-4:ctx-596c9e4b
ctx-90642717) Received unknown parameters for command
findHostsForMigration. Unknown parameters : projectid
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-20:ctx-2b17bb98
ctx-a489da47) Received unknown parameters for command
migrateVirtualMachine. Unknown parameters : projectid
WARN [c.c.a.d.ParamGenericValidationWorker]
(API-Job-Executor-55:ctx-0b76e8ae job-409 ctx-d346cca9) Received unknown
parameters for command migrateVirtualMachine. Unknown parameters : projectid
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-10:ctx-b9b78739
ctx-bc42f26e) Received unknown parameters for command queryAsyncJobResult.
Unknown parameters : projectid
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-22:ctx-22941163
ctx-737b322d) Received unknown parameters for command queryAsyncJobResult.
Unknown parameters : projectid
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-25:ctx-bd9f0a98
ctx-97c7b112) Received unknown parameters for command listNetworkOfferings.
Unknown parameters : projectid
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-16:ctx-c10f59b2
ctx-405d4cf3) Received unknown parameters for command listZones. Unknown
parameters : projectid
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-17:ctx-9bb4f185
ctx-9f2b37a9) Received unknown parameters for command listZones. Unknown
parameters : projectid
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-11:ctx-6a51a841
ctx-a0d37d54) Received unknown parameters for command listHosts. Unknown
parameters : projectid
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-22:ctx-656c8c86
ctx-aef2da97) Received unknown parameters for command listNetworkOfferings.
Unknown parameters : projectid
WARN [c.c.a.m.AgentManagerImpl] (AgentMonitor-1:ctx-9eef326e) Disconnect
agent for CPVM/SSVM due to physical connection close. host: 9
INFO [c.c.u.e.CSExceptionErrorCode] (StatsCollector-3:ctx-eafd1ed4) Could
not find exception: com.cloud.exception.OperationTimedoutException in error
code list for exceptions
WARN [c.c.a.m.AgentAttache] (StatsCollector-3:ctx-eafd1ed4) Seq
9-8454382400481264659: Timed out on null
ERROR [c.c.s.StatsCollector] (StatsCollector-3:ctx-eafd1ed4) Error trying
to retrieve storage stats
com.cloud.utils.exception.CloudRuntimeException: Failed to send command,
due to Agent:9, com.cloud.exception.OperationTimedoutException: Commands
8454382400481264659 to Host 9 timed out after 3600
WARN [o.a.c.alerts] (HA-1:ctx-6b60e29b) alertType:: 13 // dataCenterId::
0 // podId:: 0 // clusterId:: null // message:: No usage server process
running
PS: I was loggen in the vm with offer HA, and I started a ping. I lost the
connection after 30 icmp_seq.
Thanks!
On Wed, Jul 15, 2015 at 7:11 PM, Somesh Naidu <So...@citrix.com>
wrote:
> "TAG H.A checked" - did you mean "Offer HA"?
>
> Also, after you shutdown host A, what was the status of the host reported
> in Cloudstack? Also, did you check the management server logs to see what
> the investigators reported?
>
> Regards,
> Somesh
>
> -----Original Message-----
> From: Luciano Castro [mailto:luciano.castro@gmail.com]
> Sent: Wednesday, July 15, 2015 2:46 PM
> To: users@cloudstack.apache.org
> Subject: Fwd: HA feature - KVM - CloudStack 4.5.1
>
> Hi there,
>
> I´m new in the list, and I´m using 4.5.1 CloudStack version, and I
> started making some tests with H.A feature for VMs,
> but I didn´t have success. I don´t know if it´s a bug, or something that
> I´m doing wrong.
>
> Scenario: I have 1 manager server and 2 KVM hosts ('A' and 'B').
>
>
>
> 1) I created a "service offering" (computing offering) with the TAG H.A
> checked.
>
> 2) then, I created a VM with this offering, and the VM is in KVM 'A'.
>
> 3) I did a shutdown in the KVM host 'A', and I expected that this VM, after
> some minutes will be available in another KVM host ('B')
>
> But it didn´t happened.
>
> Any ideias??
>
> thanks!
>
> --
> Luciano Castro
>
>
>
> --
> Luciano Castro
>
--
Luciano Castro
Re: HA feature - KVM - CloudStack 4.5.1
Posted by Luciano Castro <lu...@gmail.com>.
Hi!
* Yes, Offer HA.
* The status os KVM hosts reported is "Disconnected", and curiously the VM
status is still "Running..."
I did the same test again:
1) shutdown KVM host 12:13 UTC
2) Logs (management server)
[root@1q2 ~]# grep -i -E 'warn|exception|unable|fail|error'
/var/log/cloudstack/management/management-server.log|tail -50
2015-07-16 12:13:21,195 WARN [c.c.a.d.ParamGenericValidationWorker]
(catalina-exec-14:ctx-b1ba1de6 ctx-5c5114f1) Received unknown parameters
for command listNetworkOfferings. Unknown parameters : projectid
2015-07-16 12:13:31,395 WARN [c.c.a.d.ParamGenericValidationWorker]
(catalina-exec-4:ctx-78cb8329 ctx-ea77f742) Received unknown parameters for
command listNetworkOfferings. Unknown parameters : projectid
2015-07-16 12:13:40,975 WARN [c.c.a.d.ParamGenericValidationWorker]
(catalina-exec-19:ctx-6e2ab11e ctx-f0bc3e8d) Received unknown parameters
for command expungeVirtualMachine. Unknown parameters : projectid
2015-07-16 12:13:40,998 WARN [c.c.a.d.ParamGenericValidationWorker]
(API-Job-Executor-54:ctx-a6765444 job-407 ctx-e7d4ae6e) Received unknown
parameters for command expungeVirtualMachine. Unknown parameters : projectid
2015-07-16 12:13:44,162 WARN [c.c.a.d.ParamGenericValidationWorker]
(catalina-exec-3:ctx-ca71fefa ctx-3a46d16f) Received unknown parameters for
command queryAsyncJobResult. Unknown parameters : projectid
2015-07-16 12:13:47,163 WARN [c.c.a.d.ParamGenericValidationWorker]
(catalina-exec-2:ctx-9773fe5c ctx-20f2c53d) Received unknown parameters for
command listNetworkOfferings. Unknown parameters : projectid
2015-07-16 12:13:51,478 WARN [c.c.a.d.ParamGenericValidationWorker]
(catalina-exec-4:ctx-986e5695 ctx-cfd9dc2a) Received unknown parameters for
command listNetworkOfferings. Unknown parameters : projectid
2015-07-16 12:13:57,139 WARN [c.c.a.d.ParamGenericValidationWorker]
(catalina-exec-21:ctx-3ed5162d ctx-0b378da3) Received unknown parameters
for command listZones. Unknown parameters : projectid
2015-07-16 12:13:58,786 WARN [c.c.a.d.ParamGenericValidationWorker]
(catalina-exec-19:ctx-ef9b46ae ctx-1ec7c374) Received unknown parameters
for command listZones. Unknown parameters : projectid
2015-07-16 12:13:58,991 WARN [c.c.a.d.ParamGenericValidationWorker]
(catalina-exec-3:ctx-c40727ff ctx-129cff5b) Received unknown parameters for
command listHosts. Unknown parameters : projectid
2015-07-16 12:14:01,369 WARN [c.c.a.d.ParamGenericValidationWorker]
(catalina-exec-9:ctx-d39d30d5 ctx-7d1296f3) Received unknown parameters for
command listHosts. Unknown parameters : projectid
2015-07-16 12:14:01,553 WARN [c.c.a.d.ParamGenericValidationWorker]
(catalina-exec-6:ctx-f02aae15 ctx-b675547d) Received unknown parameters for
command listDedicatedHosts. Unknown parameters : projectid
2015-07-16 12:14:01,725 WARN [c.c.a.d.ParamGenericValidationWorker]
(catalina-exec-25:ctx-4de9059a ctx-15ac62c3) Received unknown parameters
for command listConfigurations. Unknown parameters : projectid
2015-07-16 12:14:01,908 WARN [c.c.a.d.ParamGenericValidationWorker]
(catalina-exec-23:ctx-25fb3589 ctx-d4274e7c) Received unknown parameters
for command listOsCategories. Unknown parameters : projectid
2015-07-16 12:14:02,081 WARN [c.c.a.d.ParamGenericValidationWorker]
(catalina-exec-2:ctx-a90cf3ca ctx-6e7a01fc) Received unknown parameters for
command listHosts. Unknown parameters : projectid
2015-07-16 12:14:02,266 WARN [c.c.a.d.ParamGenericValidationWorker]
(catalina-exec-12:ctx-b8f987fa ctx-8a7f2922) Received unknown parameters
for command listDedicatedHosts. Unknown parameters : projectid
2015-07-16 12:14:02,442 WARN [c.c.a.d.ParamGenericValidationWorker]
(catalina-exec-7:ctx-64be7d4b ctx-36a9864d) Received unknown parameters for
command listConfigurations. Unknown parameters : projectid
2015-07-16 12:14:02,650 WARN [c.c.a.d.ParamGenericValidationWorker]
(catalina-exec-1:ctx-9b0043f1 ctx-36482b14) Received unknown parameters for
command listOsCategories. Unknown parameters : projectid
2015-07-16 12:14:14,981 WARN [c.c.a.d.ParamGenericValidationWorker]
(catalina-exec-16:ctx-5d62538b ctx-31cf1646) Received unknown parameters
for command listZones. Unknown parameters : projectid
2015-07-16 12:14:17,445 WARN [c.c.a.d.ParamGenericValidationWorker]
(catalina-exec-17:ctx-ef67f80f ctx-e953915f) Received unknown parameters
for command listZones. Unknown parameters : projectid
2015-07-16 12:14:17,654 WARN [c.c.a.d.ParamGenericValidationWorker]
(catalina-exec-11:ctx-7a3e6ce3 ctx-7d06a5a4) Received unknown parameters
for command listHosts. Unknown parameters : projectid
2015-07-16 12:14:24,537 WARN [c.c.a.d.ParamGenericValidationWorker]
(catalina-exec-4:ctx-596c9e4b ctx-90642717) Received unknown parameters for
command findHostsForMigration. Unknown parameters : projectid
2015-07-16 12:14:28,684 WARN [c.c.a.d.ParamGenericValidationWorker]
(catalina-exec-20:ctx-2b17bb98 ctx-a489da47) Received unknown parameters
for command migrateVirtualMachine. Unknown parameters : projectid
2015-07-16 12:14:28,708 WARN [c.c.a.d.ParamGenericValidationWorker]
(API-Job-Executor-55:ctx-0b76e8ae job-409 ctx-d346cca9) Received unknown
parameters for command migrateVirtualMachine. Unknown parameters : projectid
2015-07-16 12:14:31,880 WARN [c.c.a.d.ParamGenericValidationWorker]
(catalina-exec-10:ctx-b9b78739 ctx-bc42f26e) Received unknown parameters
for command queryAsyncJobResult. Unknown parameters : projectid
2015-07-16 12:14:34,873 WARN [c.c.a.d.ParamGenericValidationWorker]
(catalina-exec-22:ctx-22941163 ctx-737b322d) Received unknown parameters
for command queryAsyncJobResult. Unknown parameters : projectid
2015-07-16 12:15:47,758 WARN [c.c.a.d.ParamGenericValidationWorker]
(catalina-exec-25:ctx-bd9f0a98 ctx-97c7b112) Received unknown parameters
for command listNetworkOfferings. Unknown parameters : projectid
2015-07-16 12:15:50,466 WARN [c.c.a.d.ParamGenericValidationWorker]
(catalina-exec-16:ctx-c10f59b2 ctx-405d4cf3) Received unknown parameters
for command listZones. Unknown parameters : projectid
2015-07-16 12:15:54,480 WARN [c.c.a.d.ParamGenericValidationWorker]
(catalina-exec-17:ctx-9bb4f185 ctx-9f2b37a9) Received unknown parameters
for command listZones. Unknown parameters : projectid
2015-07-16 12:15:54,684 WARN [c.c.a.d.ParamGenericValidationWorker]
(catalina-exec-11:ctx-6a51a841 ctx-a0d37d54) Received unknown parameters
for command listHosts. Unknown parameters : projectid
2015-07-16 12:15:58,044 WARN [c.c.a.d.ParamGenericValidationWorker]
(catalina-exec-22:ctx-656c8c86 ctx-aef2da97) Received unknown parameters
for command listNetworkOfferings. Unknown parameters : projectid
2015-07-16 12:16:37,381 DEBUG [c.c.c.CapacityManagerImpl]
(API-Job-Executor-57:ctx-e1ed3c62 job-412 ctx-e861a024) STATS: Failed to
alloc resource from host: 3 reservedCpu: 0, requested cpu: 4000,
reservedMem: 0, requested mem: 2147483648
2015-07-16 12:21:18,239 WARN [c.c.a.m.AgentManagerImpl]
(AgentMonitor-1:ctx-9eef326e) Disconnect agent for CPVM/SSVM due to
physical connection close. host: 9
2015-07-16 12:21:18,243 INFO [c.c.u.e.CSExceptionErrorCode]
(StatsCollector-3:ctx-eafd1ed4) Could not find exception:
com.cloud.exception.OperationTimedoutException in error code list for
exceptions
2015-07-16 12:21:18,243 WARN [c.c.a.m.AgentAttache]
(StatsCollector-3:ctx-eafd1ed4) Seq 9-8454382400481264659: Timed out on null
2015-07-16 12:21:18,243 DEBUG [o.a.c.s.RemoteHostEndPoint]
(StatsCollector-3:ctx-eafd1ed4) Failed to send command, due to Agent:9,
com.cloud.exception.OperationTimedoutException: Commands
8454382400481264659 to Host 9 timed out after 3600
2015-07-16 12:21:18,243 ERROR [c.c.s.StatsCollector]
(StatsCollector-3:ctx-eafd1ed4) Error trying to retrieve storage stats
com.cloud.utils.exception.CloudRuntimeException: Failed to send command,
due to Agent:9, com.cloud.exception.OperationTimedoutException: Commands
8454382400481264659 to Host 9 timed out after 3600
2015-07-16 12:22:09,951 WARN [o.a.c.alerts] (HA-1:ctx-6b60e29b)
alertType:: 13 // dataCenterId:: 0 // podId:: 0 // clusterId:: null //
message:: No usage server process running
[root@1q2 ~]# grep -i -E 'warn|exception|unable|fail|error'
/var/log/cloudstack/management/catalina.out|tail -50
WARN [o.a.c.alerts] (HA-1:ctx-830dab68) alertType:: 13 // dataCenterId::
0 // podId:: 0 // clusterId:: null // message:: No usage server process
running
WARN [o.a.c.alerts] (HA-1:ctx-e45484a1) alertType:: 13 // dataCenterId::
0 // podId:: 0 // clusterId:: null // message:: No usage server process
running
WARN [o.a.c.alerts] (HA-1:ctx-c27a224e) alertType:: 13 // dataCenterId::
0 // podId:: 0 // clusterId:: null // message:: No usage server process
running
WARN [o.a.c.alerts] (HA-1:ctx-259fdd28) alertType:: 13 // dataCenterId::
0 // podId:: 0 // clusterId:: null // message:: No usage server process
running
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-16:ctx-40e06aca
ctx-d419dc55) Received unknown parameters for command listNetworkOfferings.
Unknown parameters : projectid
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-22:ctx-4b3562fd
ctx-3341445a) Received unknown parameters for command listZones. Unknown
parameters : projectid
WARN [o.a.c.alerts] (HA-1:ctx-f34fcc40) alertType:: 13 // dataCenterId::
0 // podId:: 0 // clusterId:: null // message:: No usage server process
running
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-19:ctx-8f8e296e
ctx-a6f598c1) Received unknown parameters for command listZones. Unknown
parameters : projectid
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-3:ctx-8825ea69
ctx-fb7312c0) Received unknown parameters for command listZones. Unknown
parameters : projectid
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-9:ctx-d6a4922e
ctx-b3bad36d) Received unknown parameters for command listHosts. Unknown
parameters : projectid
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-17:ctx-f7e28168
ctx-e966ba7a) Received unknown parameters for command listServiceOfferings.
Unknown parameters : listall
WARN [o.a.c.alerts] (HA-1:ctx-42103a07) alertType:: 13 // dataCenterId::
0 // podId:: 0 // clusterId:: null // message:: No usage server process
running
WARN [o.a.c.alerts] (HA-1:ctx-5cac1938) alertType:: 13 // dataCenterId::
0 // podId:: 0 // clusterId:: null // message:: No usage server process
running
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-14:ctx-b1ba1de6
ctx-5c5114f1) Received unknown parameters for command listNetworkOfferings.
Unknown parameters : projectid
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-4:ctx-78cb8329
ctx-ea77f742) Received unknown parameters for command listNetworkOfferings.
Unknown parameters : projectid
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-19:ctx-6e2ab11e
ctx-f0bc3e8d) Received unknown parameters for command
expungeVirtualMachine. Unknown parameters : projectid
WARN [c.c.a.d.ParamGenericValidationWorker]
(API-Job-Executor-54:ctx-a6765444 job-407 ctx-e7d4ae6e) Received unknown
parameters for command expungeVirtualMachine. Unknown parameters : projectid
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-3:ctx-ca71fefa
ctx-3a46d16f) Received unknown parameters for command queryAsyncJobResult.
Unknown parameters : projectid
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-2:ctx-9773fe5c
ctx-20f2c53d) Received unknown parameters for command listNetworkOfferings.
Unknown parameters : projectid
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-4:ctx-986e5695
ctx-cfd9dc2a) Received unknown parameters for command listNetworkOfferings.
Unknown parameters : projectid
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-21:ctx-3ed5162d
ctx-0b378da3) Received unknown parameters for command listZones. Unknown
parameters : projectid
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-19:ctx-ef9b46ae
ctx-1ec7c374) Received unknown parameters for command listZones. Unknown
parameters : projectid
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-3:ctx-c40727ff
ctx-129cff5b) Received unknown parameters for command listHosts. Unknown
parameters : projectid
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-9:ctx-d39d30d5
ctx-7d1296f3) Received unknown parameters for command listHosts. Unknown
parameters : projectid
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-6:ctx-f02aae15
ctx-b675547d) Received unknown parameters for command listDedicatedHosts.
Unknown parameters : projectid
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-25:ctx-4de9059a
ctx-15ac62c3) Received unknown parameters for command listConfigurations.
Unknown parameters : projectid
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-23:ctx-25fb3589
ctx-d4274e7c) Received unknown parameters for command listOsCategories.
Unknown parameters : projectid
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-2:ctx-a90cf3ca
ctx-6e7a01fc) Received unknown parameters for command listHosts. Unknown
parameters : projectid
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-12:ctx-b8f987fa
ctx-8a7f2922) Received unknown parameters for command listDedicatedHosts.
Unknown parameters : projectid
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-7:ctx-64be7d4b
ctx-36a9864d) Received unknown parameters for command listConfigurations.
Unknown parameters : projectid
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-1:ctx-9b0043f1
ctx-36482b14) Received unknown parameters for command listOsCategories.
Unknown parameters : projectid
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-16:ctx-5d62538b
ctx-31cf1646) Received unknown parameters for command listZones. Unknown
parameters : projectid
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-17:ctx-ef67f80f
ctx-e953915f) Received unknown parameters for command listZones. Unknown
parameters : projectid
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-11:ctx-7a3e6ce3
ctx-7d06a5a4) Received unknown parameters for command listHosts. Unknown
parameters : projectid
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-4:ctx-596c9e4b
ctx-90642717) Received unknown parameters for command
findHostsForMigration. Unknown parameters : projectid
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-20:ctx-2b17bb98
ctx-a489da47) Received unknown parameters for command
migrateVirtualMachine. Unknown parameters : projectid
WARN [c.c.a.d.ParamGenericValidationWorker]
(API-Job-Executor-55:ctx-0b76e8ae job-409 ctx-d346cca9) Received unknown
parameters for command migrateVirtualMachine. Unknown parameters : projectid
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-10:ctx-b9b78739
ctx-bc42f26e) Received unknown parameters for command queryAsyncJobResult.
Unknown parameters : projectid
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-22:ctx-22941163
ctx-737b322d) Received unknown parameters for command queryAsyncJobResult.
Unknown parameters : projectid
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-25:ctx-bd9f0a98
ctx-97c7b112) Received unknown parameters for command listNetworkOfferings.
Unknown parameters : projectid
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-16:ctx-c10f59b2
ctx-405d4cf3) Received unknown parameters for command listZones. Unknown
parameters : projectid
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-17:ctx-9bb4f185
ctx-9f2b37a9) Received unknown parameters for command listZones. Unknown
parameters : projectid
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-11:ctx-6a51a841
ctx-a0d37d54) Received unknown parameters for command listHosts. Unknown
parameters : projectid
WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-22:ctx-656c8c86
ctx-aef2da97) Received unknown parameters for command listNetworkOfferings.
Unknown parameters : projectid
WARN [c.c.a.m.AgentManagerImpl] (AgentMonitor-1:ctx-9eef326e) Disconnect
agent for CPVM/SSVM due to physical connection close. host: 9
INFO [c.c.u.e.CSExceptionErrorCode] (StatsCollector-3:ctx-eafd1ed4) Could
not find exception: com.cloud.exception.OperationTimedoutException in error
code list for exceptions
WARN [c.c.a.m.AgentAttache] (StatsCollector-3:ctx-eafd1ed4) Seq
9-8454382400481264659: Timed out on null
ERROR [c.c.s.StatsCollector] (StatsCollector-3:ctx-eafd1ed4) Error trying
to retrieve storage stats
com.cloud.utils.exception.CloudRuntimeException: Failed to send command,
due to Agent:9, com.cloud.exception.OperationTimedoutException: Commands
8454382400481264659 to Host 9 timed out after 3600
WARN [o.a.c.alerts] (HA-1:ctx-6b60e29b) alertType:: 13 // dataCenterId::
0 // podId:: 0 // clusterId:: null // message:: No usage server process
running
PS: I was loggen in the vm with offer HA, and I started a ping. I lost the
connection after 30 icmp_seq.
Thanks!
On Wed, Jul 15, 2015 at 7:11 PM, Somesh Naidu <So...@citrix.com>
wrote:
> "TAG H.A checked" - did you mean "Offer HA"?
>
> Also, after you shutdown host A, what was the status of the host reported
> in Cloudstack? Also, did you check the management server logs to see what
> the investigators reported?
>
> Regards,
> Somesh
>
> -----Original Message-----
> From: Luciano Castro [mailto:luciano.castro@gmail.com]
> Sent: Wednesday, July 15, 2015 2:46 PM
> To: users@cloudstack.apache.org
> Subject: Fwd: HA feature - KVM - CloudStack 4.5.1
>
> Hi there,
>
> I´m new in the list, and I´m using 4.5.1 CloudStack version, and I
> started making some tests with H.A feature for VMs,
> but I didn´t have success. I don´t know if it´s a bug, or something that
> I´m doing wrong.
>
> Scenario: I have 1 manager server and 2 KVM hosts ('A' and 'B').
>
>
>
> 1) I created a "service offering" (computing offering) with the TAG H.A
> checked.
>
> 2) then, I created a VM with this offering, and the VM is in KVM 'A'.
>
> 3) I did a shutdown in the KVM host 'A', and I expected that this VM, after
> some minutes will be available in another KVM host ('B')
>
> But it didn´t happened.
>
> Any ideias??
>
> thanks!
>
> --
> Luciano Castro
>
>
>
> --
> Luciano Castro
>
--
Luciano Castro
RE: HA feature - KVM - CloudStack 4.5.1
Posted by Somesh Naidu <So...@citrix.com>.
"TAG H.A checked" - did you mean "Offer HA"?
Also, after you shutdown host A, what was the status of the host reported in Cloudstack? Also, did you check the management server logs to see what the investigators reported?
Regards,
Somesh
-----Original Message-----
From: Luciano Castro [mailto:luciano.castro@gmail.com]
Sent: Wednesday, July 15, 2015 2:46 PM
To: users@cloudstack.apache.org
Subject: Fwd: HA feature - KVM - CloudStack 4.5.1
Hi there,
I´m new in the list, and I´m using 4.5.1 CloudStack version, and I
started making some tests with H.A feature for VMs,
but I didn´t have success. I don´t know if it´s a bug, or something that
I´m doing wrong.
Scenario: I have 1 manager server and 2 KVM hosts ('A' and 'B').
1) I created a "service offering" (computing offering) with the TAG H.A
checked.
2) then, I created a VM with this offering, and the VM is in KVM 'A'.
3) I did a shutdown in the KVM host 'A', and I expected that this VM, after
some minutes will be available in another KVM host ('B')
But it didn´t happened.
Any ideias??
thanks!
--
Luciano Castro
--
Luciano Castro