You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@cloudstack.apache.org by Jeff Reed <je...@jeffreed.net> on 2012/11/29 18:54:31 UTC
Zone 1 is not ready to launch console proxy yet
Hey folks,
I'm having a problem after upgrading from cloudstack 3.0.2 to 4.0. I followed the instructions here:
http://incubator.apache.org/cloudstack/docs/en-US/Apache_CloudStack/4.0.0-incubating/html/Release_Notes/upgrade-instructions.html#upgrade-from-3.0.2-to-4.0
I've upgraded the agent machines from CentOS 6.2 to CentOS 6.3 and all packages are current, and am running KVM as the hypervisor. The management server isn't running as an agent. There are two machines running as agents. I have a mishmash of hardware, but all CPUs are AMD Phenoms.
I used the convenience packages from http://cloudstack.apt-get.eu/rhel/4.0 to do the upgrade.
I have everything running, except for the system vms. Neither the console proxy nor the secondary storage VMs are running, or even seem to try to start. I can start VMs from my templates, and everything seems to work fine, but (obviously) the console proxy doesn't work, and machines that require secondary storage won't work (but I don't currently have any of those).
The related messages from the server:
2012-11-29 09:44:56,261 TRACE [db.Transaction.Connection] (consoleproxy-1:null) Creating a DB connection with no txn: for 0: dbconn1086427264. Stack: -Transaction.prepareStatement:453-Transaction.prepareAutoCloseStatement:446-GenericDaoBase.searchIncludingRemoved:380-DatabaseCallback.intercept:34-GenericDaoBase.searchIncludingRemoved:342-DatabaseCallback.intercept:34-GenericDaoBase.listIncludingRemovedBy:894-GenericDaoBase.listBy:871-GenericDaoBase.listBy:884-DataCenterDaoImpl.listEnabledZones:366-DatabaseCallback.intercept:34-ConsoleProxyManagerImpl.getScannablePools:1915
2012-11-29 09:44:56,261 TRACE [db.Transaction.Statement] (consoleproxy-1:null) Preparing: SELECT data_center.id, data_center.name, data_center.description, data_center.dns1, data_center.dns2, data_center.internal_dns1, data_center.internal_dns2, data_center.router_mac_address, data_center.guest_network_cidr, data_center.domain_id, data_center.domain, data_center.networktype, data_center.dns_provider, data_center.dhcp_provider, data_center.gateway_provider, data_center.vpn_provider, data_center.userdata_provider, data_center.lb_provider, data_center.firewall_provider, data_center.mac_address, data_center.zone_token, data_center.removed, data_center.allocation_state, data_center.uuid, data_center.is_security_group_enabled, data_center.is_local_storage_enabled FROM data_center WHERE data_center.allocation_state = ? AND data_center.removed IS NULL
2012-11-29 09:44:56,264 TRACE [db.Transaction.Statement] (consoleproxy-1:null) Closing: org.apache.commons.dbcp.DelegatingPreparedStatement@63f5e4b6
2012-11-29 09:44:56,264 TRACE [db.Transaction.Connection] (consoleproxy-1:null) Closing DB connection: dbconn1086427264
2012-11-29 09:44:56,264 DEBUG [cloud.consoleproxy.ConsoleProxyManagerImpl] (consoleproxy-1:null) Zone 1 is not ready to launch console proxy yet
2012-11-29 09:44:56,265 TRACE [utils.db.GlobalLock] (consoleproxy-1:null) lock consoleproxy.scan.lock is returned to free state, total holding time :32
2012-11-29 09:44:56,265 TRACE [utils.db.GlobalLock] (consoleproxy-1:null) lock consoleproxy.scan.lock is released, lock count :0
2012-11-29 09:44:56,267 TRACE [utils.db.GenericDaoBase] (consoleproxy-1:null) Select SQL: SELECT stack_maid.id, stack_maid.msid, stack_maid.thread_id, stack_maid.seq, stack_maid.cleanup_delegate, stack_maid.cleanup_context, stack_maid.created FROM stack_maid WHERE
2012-11-29 09:44:56,267 TRACE [utils.db.GenericDaoBase] (consoleproxy-1:null) Remove SQL: No remove sql
2012-11-29 09:44:56,267 TRACE [utils.db.GenericDaoBase] (consoleproxy-1:null) Select by Id SQL: SELECT stack_maid.id, stack_maid.msid, stack_maid.thread_id, stack_maid.seq, stack_maid.cleanup_delegate, stack_maid.cleanup_context, stack_maid.created FROM stack_maid WHERE stack_maid.id = ?
The systemVM was downloaded from http://download.cloud.com/templates/acton/acton-systemvm-02062012.qcow2.bz2 when my cloud was originally set up (which incidentally is the same as given in the installation instructions for 4.0-incubator).
The system VM shows in the database as being DOWNLOADED:
id: 3
host_id: 2
template_id: 3
created: 2012-10-24 23:56:42
last_updated: 2012-11-27 12:04:05
job_id: NULL
download_pct: 100
size: 725811200
physical_size: 725811200
download_state: DOWNLOADED
error_str: NULL
local_path: NULL
install_path: template/tmpl/1/3//b5d17e02-394a-4a01-bf13-6963b3127c39.qcow2
url: http://download.cloud.com/templates/acton/acton-systemvm-02062012.qcow2.bz2
destroyed: 0
is_copy: 0
My 3.0.2 CS installation ran perfectly since I set it up, but I'd really like to get 4.0-incubator up and running correctly.
I've googled and tried pretty much everything I've found (which isn't much), but to no avail. I can't find any reason that the management server thinks the zone isn't ready to launch the system VMs. I've restarted the pod, the cluster, the hosts in the cluster, the zone, but I still have the issue.
Any help would be appreciated.
Jeff
Re: Zone 1 is not ready to launch console proxy yet
Posted by Dave Cundiff <sy...@gmail.com>.
Hi,
I had the same issue after a 3.0.3 upgrade. I erased my system VMs to
rebuild and they were never re-created. I dug through the MySQL queries and
code a bit. The closest I got was it appeared to be the code checking to
see if the zone was ready. It saw mine as not ready and I could never
figure out why.
It was a test system so I finally gave up and just reset the database using
the 4.0 tools. Still should probably be looked into since others may run
into the same issue.
On Fri, Nov 30, 2012 at 5:26 AM, Tamas Monos <ta...@veber.co.uk> wrote:
> Hi,
>
> I have the same here.
> Additionally if I create a new account and want to deploy the first
> instance on the new account then it always fails with a MySQL duplicate
> entry exception.
> The management server calls the "Adding DB entries" bit twice and the
> second throws an exception and rolls everything back.
> I'd like to use 4.0 especially because of the console proxy can follow
> screen resolution changes, but it seems I cannot go for 4.0 just yet.
> I've filed several Critical and Blocker issues for 4.0
>
> Regards
>
> Tamas Monos DDI
> +44(0)2034687012
> Chief Technical Office
> +44(0)2034687000
> Veber: The Hosting Specialists Fax +44(0)871 522
> 7057
> http://www.veber.co.uk
>
> Follow us on Twitter: www.twitter.com/veberhost
> Follow us on Facebook: www.facebook.com/veberhost
>
> -----Original Message-----
> From: Jeff Reed [mailto:jeffreed@jeffreed.net]
> Sent: 29 November 2012 17:55
> To: cloudstack-users@incubator.apache.org
> Subject: Zone 1 is not ready to launch console proxy yet
>
> Hey folks,
>
> I'm having a problem after upgrading from cloudstack 3.0.2 to 4.0. I
> followed the instructions here:
>
> http://incubator.apache.org/cloudstack/docs/en-US/Apache_CloudStack/4.0.0-incubating/html/Release_Notes/upgrade-instructions.html#upgrade-from-3.0.2-to-4.0
>
> I've upgraded the agent machines from CentOS 6.2 to CentOS 6.3 and all
> packages are current, and am running KVM as the hypervisor. The management
> server isn't running as an agent. There are two machines running as
> agents. I have a mishmash of hardware, but all CPUs are AMD Phenoms.
>
> I used the convenience packages from http://cloudstack.apt-get.eu/rhel/4.0to do the upgrade.
>
> I have everything running, except for the system vms. Neither the console
> proxy nor the secondary storage VMs are running, or even seem to try to
> start. I can start VMs from my templates, and everything seems to work
> fine, but (obviously) the console proxy doesn't work, and machines that
> require secondary storage won't work (but I don't currently have any of
> those).
>
> The related messages from the server:
> 2012-11-29 09:44:56,261 TRACE [db.Transaction.Connection]
> (consoleproxy-1:null) Creating a DB connection with no txn: for 0:
> dbconn1086427264. Stack:
> -Transaction.prepareStatement:453-Transaction.prepareAutoCloseStatement:446-GenericDaoBase.searchIncludingRemoved:380-DatabaseCallback.intercept:34-GenericDaoBase.searchIncludingRemoved:342-DatabaseCallback.intercept:34-GenericDaoBase.listIncludingRemovedBy:894-GenericDaoBase.listBy:871-GenericDaoBase.listBy:884-DataCenterDaoImpl.listEnabledZones:366-DatabaseCallback.intercept:34-ConsoleProxyManagerImpl.getScannablePools:1915
> 2012-11-29 09:44:56,261 TRACE [db.Transaction.Statement]
> (consoleproxy-1:null) Preparing: SELECT data_center.id, data_center.name,
> data_center.description, data_center.dns1, data_center.dns2,
> data_center.internal_dns1, data_center.internal_dns2,
> data_center.router_mac_address, data_center.guest_network_cidr,
> data_center.domain_id, data_center.domain, data_center.networktype,
> data_center.dns_provider, data_center.dhcp_provider,
> data_center.gateway_provider, data_center.vpn_provider,
> data_center.userdata_provider, data_center.lb_provider,
> data_center.firewall_provider, data_center.mac_address,
> data_center.zone_token, data_center.removed, data_center.allocation_state,
> data_center.uuid, data_center.is_security_group_enabled,
> data_center.is_local_storage_enabled FROM data_center WHERE
> data_center.allocation_state = ? AND data_center.removed IS NULL
> 2012-11-29 09:44:56,264 TRACE [db.Transaction.Statement]
> (consoleproxy-1:null) Closing:
> org.apache.commons.dbcp.DelegatingPreparedStatement@63f5e4b6
> 2012-11-29 09:44:56,264 TRACE [db.Transaction.Connection]
> (consoleproxy-1:null) Closing DB connection: dbconn1086427264
> 2012-11-29 09:44:56,264 DEBUG [cloud.consoleproxy.ConsoleProxyManagerImpl]
> (consoleproxy-1:null) Zone 1 is not ready to launch console proxy yet
> 2012-11-29 09:44:56,265 TRACE [utils.db.GlobalLock] (consoleproxy-1:null)
> lock consoleproxy.scan.lock is returned to free state, total holding time
> :32
> 2012-11-29 09:44:56,265 TRACE [utils.db.GlobalLock] (consoleproxy-1:null)
> lock consoleproxy.scan.lock is released, lock count :0
> 2012-11-29 09:44:56,267 TRACE [utils.db.GenericDaoBase]
> (consoleproxy-1:null) Select SQL: SELECT stack_maid.id, stack_maid.msid,
> stack_maid.thread_id, stack_maid.seq, stack_maid.cleanup_delegate,
> stack_maid.cleanup_context, stack_maid.created FROM stack_maid WHERE
> 2012-11-29 09:44:56,267 TRACE [utils.db.GenericDaoBase]
> (consoleproxy-1:null) Remove SQL: No remove sql
> 2012-11-29 09:44:56,267 TRACE [utils.db.GenericDaoBase]
> (consoleproxy-1:null) Select by Id SQL: SELECT stack_maid.id,
> stack_maid.msid, stack_maid.thread_id, stack_maid.seq,
> stack_maid.cleanup_delegate, stack_maid.cleanup_context, stack_maid.created
> FROM stack_maid WHERE stack_maid.id = ?
>
>
> The systemVM was downloaded from
> http://download.cloud.com/templates/acton/acton-systemvm-02062012.qcow2.bz2when my cloud was originally set up (which incidentally is the same as
> given in the installation instructions for 4.0-incubator).
>
> The system VM shows in the database as being DOWNLOADED:
> id: 3
> host_id: 2
> template_id: 3
> created: 2012-10-24 23:56:42
> last_updated: 2012-11-27 12:04:05
> job_id: NULL
> download_pct: 100
> size: 725811200
> physical_size: 725811200
> download_state: DOWNLOADED
> error_str: NULL
> local_path: NULL
> install_path:
> template/tmpl/1/3//b5d17e02-394a-4a01-bf13-6963b3127c39.qcow2
> url:
> http://download.cloud.com/templates/acton/acton-systemvm-02062012.qcow2.bz2
> destroyed: 0
> is_copy: 0
>
>
> My 3.0.2 CS installation ran perfectly since I set it up, but I'd really
> like to get 4.0-incubator up and running correctly.
>
> I've googled and tried pretty much everything I've found (which isn't
> much), but to no avail. I can't find any reason that the management server
> thinks the zone isn't ready to launch the system VMs. I've restarted the
> pod, the cluster, the hosts in the cluster, the zone, but I still have the
> issue.
>
> Any help would be appreciated.
>
> Jeff
>
>
--
Dave Cundiff
System Administrator
A2Hosting, Inc
http://www.a2hosting.com
RE: Zone 1 is not ready to launch console proxy yet
Posted by Tamas Monos <ta...@veber.co.uk>.
Hi,
I have the same here.
Additionally if I create a new account and want to deploy the first instance on the new account then it always fails with a MySQL duplicate entry exception.
The management server calls the "Adding DB entries" bit twice and the second throws an exception and rolls everything back.
I'd like to use 4.0 especially because of the console proxy can follow screen resolution changes, but it seems I cannot go for 4.0 just yet.
I've filed several Critical and Blocker issues for 4.0
Regards
Tamas Monos DDI +44(0)2034687012
Chief Technical Office +44(0)2034687000
Veber: The Hosting Specialists Fax +44(0)871 522 7057
http://www.veber.co.uk
Follow us on Twitter: www.twitter.com/veberhost
Follow us on Facebook: www.facebook.com/veberhost
-----Original Message-----
From: Jeff Reed [mailto:jeffreed@jeffreed.net]
Sent: 29 November 2012 17:55
To: cloudstack-users@incubator.apache.org
Subject: Zone 1 is not ready to launch console proxy yet
Hey folks,
I'm having a problem after upgrading from cloudstack 3.0.2 to 4.0. I followed the instructions here:
http://incubator.apache.org/cloudstack/docs/en-US/Apache_CloudStack/4.0.0-incubating/html/Release_Notes/upgrade-instructions.html#upgrade-from-3.0.2-to-4.0
I've upgraded the agent machines from CentOS 6.2 to CentOS 6.3 and all packages are current, and am running KVM as the hypervisor. The management server isn't running as an agent. There are two machines running as agents. I have a mishmash of hardware, but all CPUs are AMD Phenoms.
I used the convenience packages from http://cloudstack.apt-get.eu/rhel/4.0 to do the upgrade.
I have everything running, except for the system vms. Neither the console proxy nor the secondary storage VMs are running, or even seem to try to start. I can start VMs from my templates, and everything seems to work fine, but (obviously) the console proxy doesn't work, and machines that require secondary storage won't work (but I don't currently have any of those).
The related messages from the server:
2012-11-29 09:44:56,261 TRACE [db.Transaction.Connection] (consoleproxy-1:null) Creating a DB connection with no txn: for 0: dbconn1086427264. Stack: -Transaction.prepareStatement:453-Transaction.prepareAutoCloseStatement:446-GenericDaoBase.searchIncludingRemoved:380-DatabaseCallback.intercept:34-GenericDaoBase.searchIncludingRemoved:342-DatabaseCallback.intercept:34-GenericDaoBase.listIncludingRemovedBy:894-GenericDaoBase.listBy:871-GenericDaoBase.listBy:884-DataCenterDaoImpl.listEnabledZones:366-DatabaseCallback.intercept:34-ConsoleProxyManagerImpl.getScannablePools:1915
2012-11-29 09:44:56,261 TRACE [db.Transaction.Statement] (consoleproxy-1:null) Preparing: SELECT data_center.id, data_center.name, data_center.description, data_center.dns1, data_center.dns2, data_center.internal_dns1, data_center.internal_dns2, data_center.router_mac_address, data_center.guest_network_cidr, data_center.domain_id, data_center.domain, data_center.networktype, data_center.dns_provider, data_center.dhcp_provider, data_center.gateway_provider, data_center.vpn_provider, data_center.userdata_provider, data_center.lb_provider, data_center.firewall_provider, data_center.mac_address, data_center.zone_token, data_center.removed, data_center.allocation_state, data_center.uuid, data_center.is_security_group_enabled, data_center.is_local_storage_enabled FROM data_center WHERE data_center.allocation_state = ? AND data_center.removed IS NULL
2012-11-29 09:44:56,264 TRACE [db.Transaction.Statement] (consoleproxy-1:null) Closing: org.apache.commons.dbcp.DelegatingPreparedStatement@63f5e4b6
2012-11-29 09:44:56,264 TRACE [db.Transaction.Connection] (consoleproxy-1:null) Closing DB connection: dbconn1086427264
2012-11-29 09:44:56,264 DEBUG [cloud.consoleproxy.ConsoleProxyManagerImpl] (consoleproxy-1:null) Zone 1 is not ready to launch console proxy yet
2012-11-29 09:44:56,265 TRACE [utils.db.GlobalLock] (consoleproxy-1:null) lock consoleproxy.scan.lock is returned to free state, total holding time :32
2012-11-29 09:44:56,265 TRACE [utils.db.GlobalLock] (consoleproxy-1:null) lock consoleproxy.scan.lock is released, lock count :0
2012-11-29 09:44:56,267 TRACE [utils.db.GenericDaoBase] (consoleproxy-1:null) Select SQL: SELECT stack_maid.id, stack_maid.msid, stack_maid.thread_id, stack_maid.seq, stack_maid.cleanup_delegate, stack_maid.cleanup_context, stack_maid.created FROM stack_maid WHERE
2012-11-29 09:44:56,267 TRACE [utils.db.GenericDaoBase] (consoleproxy-1:null) Remove SQL: No remove sql
2012-11-29 09:44:56,267 TRACE [utils.db.GenericDaoBase] (consoleproxy-1:null) Select by Id SQL: SELECT stack_maid.id, stack_maid.msid, stack_maid.thread_id, stack_maid.seq, stack_maid.cleanup_delegate, stack_maid.cleanup_context, stack_maid.created FROM stack_maid WHERE stack_maid.id = ?
The systemVM was downloaded from http://download.cloud.com/templates/acton/acton-systemvm-02062012.qcow2.bz2 when my cloud was originally set up (which incidentally is the same as given in the installation instructions for 4.0-incubator).
The system VM shows in the database as being DOWNLOADED:
id: 3
host_id: 2
template_id: 3
created: 2012-10-24 23:56:42
last_updated: 2012-11-27 12:04:05
job_id: NULL
download_pct: 100
size: 725811200
physical_size: 725811200
download_state: DOWNLOADED
error_str: NULL
local_path: NULL
install_path: template/tmpl/1/3//b5d17e02-394a-4a01-bf13-6963b3127c39.qcow2
url: http://download.cloud.com/templates/acton/acton-systemvm-02062012.qcow2.bz2
destroyed: 0
is_copy: 0
My 3.0.2 CS installation ran perfectly since I set it up, but I'd really like to get 4.0-incubator up and running correctly.
I've googled and tried pretty much everything I've found (which isn't much), but to no avail. I can't find any reason that the management server thinks the zone isn't ready to launch the system VMs. I've restarted the pod, the cluster, the hosts in the cluster, the zone, but I still have the issue.
Any help would be appreciated.
Jeff
Re: Zone 1 is not ready to launch console proxy yet
Posted by Jeff Reed <je...@jeffreed.net>.
mysql> select * from configuration where name='secondary.storage.vm';
+----------+----------+-------------------+----------------------+--------------------------+------------------------------------------------------------------------------------------------------------------------+
| category | instance | component | name | value | description |
+----------+----------+-------------------+----------------------+--------------------------+------------------------------------------------------------------------------------------------------------------------+
| Hidden | DEFAULT | management-server | secondary.storage.vm | BtVjVxSr0TZkH4QM+NIGGA== | Deploys a VM per zone to manage secondary storage if true, otherwise secondary storage is mounted on management server |
+----------+----------+-------------------+----------------------+--------------------------+------------------------------------------------------------------------------------------------------------------------+
On Dec 4, 2012, at 1:10 PM, Chiradeep Vittal <Ch...@citrix.com> wrote:
> Can you check the value of secondary.storage.vm in the configuration table?
> It should exist and its value should be 'true'.
>
>
>
> On 12/4/12 4:45 AM, "Jeff Reed" <je...@jeffreed.net> wrote:
>
>> Thanks Chiradeep for your helpŠ here are the results from both queries:
>>
>> mysql> SELECT data_center.id, data_center.name, data_center.description,
>> data_center.dns1, data_center.dns2, data_center.internal_dns1,
>> data_center.internal_dns2, data_center.router_mac_address,
>> data_center.guest_network_cidr, data_center.domain_id,
>> data_center.domain, data_center.networktype, data_center.dns_provider,
>> data_center.dhcp_provider, data_center.gateway_provider,
>> data_center.vpn_provider, data_center.userdata_provider,
>> data_center.lb_provider, data_center.firewall_provider,
>> data_center.mac_address, data_center.zone_token, data_center.removed,
>> data_center.allocation_state, data_center.uuid,
>> data_center.is_security_group_enabled,
>> data_center.is_local_storage_enabled FROM data_center WHERE
>> data_center.allocation_state = 'Enabled' AND data_center.removed IS NULL
>> \G
>> *************************** 1. row ***************************
>> id: 1
>> name: zone
>> description: NULL
>> dns1: 192.168.1.1
>> dns2: NULL
>> internal_dns1: 192.168.1.1
>> internal_dns2: NULL
>> router_mac_address: 02:00:00:00:00:01
>> guest_network_cidr: NULL
>> domain_id: NULL
>> domain: NULL
>> networktype: Basic
>> dns_provider: VirtualRouter
>> dhcp_provider: VirtualRouter
>> gateway_provider: NULL
>> vpn_provider: NULL
>> userdata_provider: VirtualRouter
>> lb_provider: ElasticLoadBalancerVm
>> firewall_provider: NULL
>> mac_address: 117
>> zone_token: b862f66d-e1de-3992-bb5c-7d8bb9811769
>> removed: NULL
>> allocation_state: Enabled
>> uuid: 55450c92-4fdc-4933-a777-fb9b9f6ab243
>> is_security_group_enabled: 1
>> is_local_storage_enabled: 0
>> 1 row in set (0.00 sec)
>>
>>
>> mysql> select * from (select h.data_center_id, h.type, count(*) as count
>> from host as h INNER JOIN mshost as m ON h.mgmt_server_id=m.msid where
>> h.status='Up' and h.type='SecondaryStorage' and m.last_update >
>> '2012-12-04 12:39:26' group by h.data_center_id, h.type UNION ALL select
>> h.data_center_id, h.type, count(*) as count from host as h INNER JOIN
>> mshost as m ON h.mgmt_server_id=m.msid where h.status='Up' and
>> h.type='Routing' and m.last_update > '2012-12-04 12:39:26' group by
>> h.data_center_id, h.type) as t ORDER by t.data_center_id, t.type;
>> +----------------+---------+-------+
>> | data_center_id | type | count |
>> +----------------+---------+-------+
>> | 1 | Routing | 2 |
>> +----------------+---------+-------+
>> 1 row in set (0.00 sec)
>>
>>
>> On Dec 3, 2012, at 10:12 PM, Chiradeep Vittal
>> <Ch...@citrix.com> wrote:
>>
>>> There's 3 minimum requirements to start a system vm:
>>> 1. The zone is enabled
>>> 2. There is primary storage ready that matches the type of the system vm
>>> offering (e.g., if only local is available, then system vm disk offering
>>> also needs to be local)
>>> 3. The system vm templates are DOWNLOADED
>>> 3. There are enough hosts in the 'UP' state
>>> select * from (select h.data_center_id, h.type, count(*) as count from
>>> host as h INNER JOIN mshost as m ON h.mgmt_server_id=m.msid
>>> where h.status='Up' and h.type='SecondaryStorage' and m.last_update > ?
>>> group by h.data_center_id, h.type UNION ALL select h.data_center_id,
>>> h.type, count(*) as count from host as h INNER JOIN mshost as m ON
>>> h.mgmt_server_id=m.msid where h.status='Up' and h.type='Routing' and
>>> m.last_update > ? group by h.data_center_id, h.type) as t ORDER by
>>> t.data_center_id, t.type;
>>>
>>> The '?' is to be replaced by a date equivalent to now() in GMT - 150
>>> secs
>>>
>>>
>>>
>>> On 12/3/12 6:58 PM, "Chiradeep Vittal" <Ch...@citrix.com>
>>> wrote:
>>>
>>>> So what is the result of the below SQL query if you run it by hand?
>>>> Use data_center.allocation_state = 'Enabled'
>>>>
>>>>
>>>> On 11/29/12 9:54 AM, "Jeff Reed" <je...@jeffreed.net> wrote:
>>>>
>>>>> Hey folks,
>>>>>
>>>>> I'm having a problem after upgrading from cloudstack 3.0.2 to 4.0. I
>>>>> followed the instructions here:
>>>>>
>>>>> http://incubator.apache.org/cloudstack/docs/en-US/Apache_CloudStack/4.0
>>>>> .0
>>>>> -
>>>>>
>>>>> incubating/html/Release_Notes/upgrade-instructions.html#upgrade-from-3.
>>>>> 0.
>>>>> 2
>>>>> -to-4.0
>>>>>
>>>>> I've upgraded the agent machines from CentOS 6.2 to CentOS 6.3 and all
>>>>> packages are current, and am running KVM as the hypervisor. The
>>>>> management server isn't running as an agent. There are two machines
>>>>> running as agents. I have a mishmash of hardware, but all CPUs are
>>>>> AMD
>>>>> Phenoms.
>>>>>
>>>>> I used the convenience packages from
>>>>> http://cloudstack.apt-get.eu/rhel/4.0 to do the upgrade.
>>>>>
>>>>> I have everything running, except for the system vms. Neither the
>>>>> console proxy nor the secondary storage VMs are running, or even seem
>>>>> to
>>>>> try to start. I can start VMs from my templates, and everything
>>>>> seems to
>>>>> work fine, but (obviously) the console proxy doesn't work, and
>>>>> machines
>>>>> that require secondary storage won't work (but I don't currently have
>>>>> any
>>>>> of those).
>>>>>
>>>>> The related messages from the server:
>>>>> 2012-11-29 09:44:56,261 TRACE [db.Transaction.Connection]
>>>>> (consoleproxy-1:null) Creating a DB connection with no txn: for 0:
>>>>> dbconn1086427264. Stack:
>>>>>
>>>>> -Transaction.prepareStatement:453-Transaction.prepareAutoCloseStatement
>>>>> :4
>>>>> 4
>>>>>
>>>>> 6-GenericDaoBase.searchIncludingRemoved:380-DatabaseCallback.intercept:
>>>>> 34
>>>>> -
>>>>>
>>>>> GenericDaoBase.searchIncludingRemoved:342-DatabaseCallback.intercept:34
>>>>> -G
>>>>> e
>>>>>
>>>>> nericDaoBase.listIncludingRemovedBy:894-GenericDaoBase.listBy:871-Gener
>>>>> ic
>>>>> D
>>>>>
>>>>> aoBase.listBy:884-DataCenterDaoImpl.listEnabledZones:366-DatabaseCallba
>>>>> ck
>>>>> .
>>>>> intercept:34-ConsoleProxyManagerImpl.getScannablePools:1915
>>>>> 2012-11-29 09:44:56,261 TRACE [db.Transaction.Statement]
>>>>> (consoleproxy-1:null) Preparing: SELECT data_center.id,
>>>>> data_center.name,
>>>>> data_center.description, data_center.dns1, data_center.dns2,
>>>>> data_center.internal_dns1, data_center.internal_dns2,
>>>>> data_center.router_mac_address, data_center.guest_network_cidr,
>>>>> data_center.domain_id, data_center.domain, data_center.networktype,
>>>>> data_center.dns_provider, data_center.dhcp_provider,
>>>>> data_center.gateway_provider, data_center.vpn_provider,
>>>>> data_center.userdata_provider, data_center.lb_provider,
>>>>> data_center.firewall_provider, data_center.mac_address,
>>>>> data_center.zone_token, data_center.removed,
>>>>> data_center.allocation_state, data_center.uuid,
>>>>> data_center.is_security_group_enabled,
>>>>> data_center.is_local_storage_enabled FROM data_center WHERE
>>>>> data_center.allocation_state = ? AND data_center.removed IS NULL
>>>>> 2012-11-29 09:44:56,264 TRACE [db.Transaction.Statement]
>>>>> (consoleproxy-1:null) Closing:
>>>>> org.apache.commons.dbcp.DelegatingPreparedStatement@63f5e4b6
>>>>> 2012-11-29 09:44:56,264 TRACE [db.Transaction.Connection]
>>>>> (consoleproxy-1:null) Closing DB connection: dbconn1086427264
>>>>> 2012-11-29 09:44:56,264 DEBUG
>>>>> [cloud.consoleproxy.ConsoleProxyManagerImpl] (consoleproxy-1:null)
>>>>> Zone 1
>>>>> is not ready to launch console proxy yet
>>>>> 2012-11-29 09:44:56,265 TRACE [utils.db.GlobalLock]
>>>>> (consoleproxy-1:null)
>>>>> lock consoleproxy.scan.lock is returned to free state, total holding
>>>>> time
>>>>> :32
>>>>> 2012-11-29 09:44:56,265 TRACE [utils.db.GlobalLock]
>>>>> (consoleproxy-1:null)
>>>>> lock consoleproxy.scan.lock is released, lock count :0
>>>>> 2012-11-29 09:44:56,267 TRACE [utils.db.GenericDaoBase]
>>>>> (consoleproxy-1:null) Select SQL: SELECT stack_maid.id,
>>>>> stack_maid.msid,
>>>>> stack_maid.thread_id, stack_maid.seq, stack_maid.cleanup_delegate,
>>>>> stack_maid.cleanup_context, stack_maid.created FROM stack_maid WHERE
>>>>> 2012-11-29 09:44:56,267 TRACE [utils.db.GenericDaoBase]
>>>>> (consoleproxy-1:null) Remove SQL: No remove sql
>>>>> 2012-11-29 09:44:56,267 TRACE [utils.db.GenericDaoBase]
>>>>> (consoleproxy-1:null) Select by Id SQL: SELECT stack_maid.id,
>>>>> stack_maid.msid, stack_maid.thread_id, stack_maid.seq,
>>>>> stack_maid.cleanup_delegate, stack_maid.cleanup_context,
>>>>> stack_maid.created FROM stack_maid WHERE stack_maid.id = ?
>>>>>
>>>>>
>>>>> The systemVM was downloaded from
>>>>>
>>>>> http://download.cloud.com/templates/acton/acton-systemvm-02062012.qcow2
>>>>> .b
>>>>> z
>>>>> 2 when my cloud was originally set up (which incidentally is the same
>>>>> as
>>>>> given in the installation instructions for 4.0-incubator).
>>>>>
>>>>> The system VM shows in the database as being DOWNLOADED:
>>>>> id: 3
>>>>> host_id: 2
>>>>> template_id: 3
>>>>> created: 2012-10-24 23:56:42
>>>>> last_updated: 2012-11-27 12:04:05
>>>>> job_id: NULL
>>>>> download_pct: 100
>>>>> size: 725811200
>>>>> physical_size: 725811200
>>>>> download_state: DOWNLOADED
>>>>> error_str: NULL
>>>>> local_path: NULL
>>>>> install_path:
>>>>> template/tmpl/1/3//b5d17e02-394a-4a01-bf13-6963b3127c39.qcow2
>>>>> url:
>>>>>
>>>>> http://download.cloud.com/templates/acton/acton-systemvm-02062012.qcow2
>>>>> .b
>>>>> z
>>>>> 2
>>>>> destroyed: 0
>>>>> is_copy: 0
>>>>>
>>>>>
>>>>> My 3.0.2 CS installation ran perfectly since I set it up, but I'd
>>>>> really
>>>>> like to get 4.0-incubator up and running correctly.
>>>>>
>>>>> I've googled and tried pretty much everything I've found (which isn't
>>>>> much), but to no avail. I can't find any reason that the management
>>>>> server thinks the zone isn't ready to launch the system VMs. I've
>>>>> restarted the pod, the cluster, the hosts in the cluster, the zone,
>>>>> but I
>>>>> still have the issue.
>>>>>
>>>>> Any help would be appreciated.
>>>>>
>>>>> Jeff
>>>>
>>>
>>
>
Re: Zone 1 is not ready to launch console proxy yet
Posted by Chiradeep Vittal <Ch...@citrix.com>.
Can you check the value of secondary.storage.vm in the configuration table?
It should exist and its value should be 'true'.
On 12/4/12 4:45 AM, "Jeff Reed" <je...@jeffreed.net> wrote:
>Thanks Chiradeep for your helpŠ here are the results from both queries:
>
>mysql> SELECT data_center.id, data_center.name, data_center.description,
>data_center.dns1, data_center.dns2, data_center.internal_dns1,
>data_center.internal_dns2, data_center.router_mac_address,
>data_center.guest_network_cidr, data_center.domain_id,
>data_center.domain, data_center.networktype, data_center.dns_provider,
>data_center.dhcp_provider, data_center.gateway_provider,
>data_center.vpn_provider, data_center.userdata_provider,
>data_center.lb_provider, data_center.firewall_provider,
>data_center.mac_address, data_center.zone_token, data_center.removed,
>data_center.allocation_state, data_center.uuid,
>data_center.is_security_group_enabled,
>data_center.is_local_storage_enabled FROM data_center WHERE
>data_center.allocation_state = 'Enabled' AND data_center.removed IS NULL
>\G
>*************************** 1. row ***************************
> id: 1
> name: zone
> description: NULL
> dns1: 192.168.1.1
> dns2: NULL
> internal_dns1: 192.168.1.1
> internal_dns2: NULL
> router_mac_address: 02:00:00:00:00:01
> guest_network_cidr: NULL
> domain_id: NULL
> domain: NULL
> networktype: Basic
> dns_provider: VirtualRouter
> dhcp_provider: VirtualRouter
> gateway_provider: NULL
> vpn_provider: NULL
> userdata_provider: VirtualRouter
> lb_provider: ElasticLoadBalancerVm
> firewall_provider: NULL
> mac_address: 117
> zone_token: b862f66d-e1de-3992-bb5c-7d8bb9811769
> removed: NULL
> allocation_state: Enabled
> uuid: 55450c92-4fdc-4933-a777-fb9b9f6ab243
>is_security_group_enabled: 1
> is_local_storage_enabled: 0
>1 row in set (0.00 sec)
>
>
>mysql> select * from (select h.data_center_id, h.type, count(*) as count
>from host as h INNER JOIN mshost as m ON h.mgmt_server_id=m.msid where
>h.status='Up' and h.type='SecondaryStorage' and m.last_update >
>'2012-12-04 12:39:26' group by h.data_center_id, h.type UNION ALL select
>h.data_center_id, h.type, count(*) as count from host as h INNER JOIN
>mshost as m ON h.mgmt_server_id=m.msid where h.status='Up' and
>h.type='Routing' and m.last_update > '2012-12-04 12:39:26' group by
>h.data_center_id, h.type) as t ORDER by t.data_center_id, t.type;
>+----------------+---------+-------+
>| data_center_id | type | count |
>+----------------+---------+-------+
>| 1 | Routing | 2 |
>+----------------+---------+-------+
>1 row in set (0.00 sec)
>
>
>On Dec 3, 2012, at 10:12 PM, Chiradeep Vittal
><Ch...@citrix.com> wrote:
>
>> There's 3 minimum requirements to start a system vm:
>> 1. The zone is enabled
>> 2. There is primary storage ready that matches the type of the system vm
>> offering (e.g., if only local is available, then system vm disk offering
>> also needs to be local)
>> 3. The system vm templates are DOWNLOADED
>> 3. There are enough hosts in the 'UP' state
>> select * from (select h.data_center_id, h.type, count(*) as count from
>> host as h INNER JOIN mshost as m ON h.mgmt_server_id=m.msid
>> where h.status='Up' and h.type='SecondaryStorage' and m.last_update > ?
>> group by h.data_center_id, h.type UNION ALL select h.data_center_id,
>> h.type, count(*) as count from host as h INNER JOIN mshost as m ON
>> h.mgmt_server_id=m.msid where h.status='Up' and h.type='Routing' and
>> m.last_update > ? group by h.data_center_id, h.type) as t ORDER by
>> t.data_center_id, t.type;
>>
>> The '?' is to be replaced by a date equivalent to now() in GMT - 150
>>secs
>>
>>
>>
>> On 12/3/12 6:58 PM, "Chiradeep Vittal" <Ch...@citrix.com>
>>wrote:
>>
>>> So what is the result of the below SQL query if you run it by hand?
>>> Use data_center.allocation_state = 'Enabled'
>>>
>>>
>>> On 11/29/12 9:54 AM, "Jeff Reed" <je...@jeffreed.net> wrote:
>>>
>>>> Hey folks,
>>>>
>>>> I'm having a problem after upgrading from cloudstack 3.0.2 to 4.0. I
>>>> followed the instructions here:
>>>>
>>>>http://incubator.apache.org/cloudstack/docs/en-US/Apache_CloudStack/4.0
>>>>.0
>>>> -
>>>>
>>>>incubating/html/Release_Notes/upgrade-instructions.html#upgrade-from-3.
>>>>0.
>>>> 2
>>>> -to-4.0
>>>>
>>>> I've upgraded the agent machines from CentOS 6.2 to CentOS 6.3 and all
>>>> packages are current, and am running KVM as the hypervisor. The
>>>> management server isn't running as an agent. There are two machines
>>>> running as agents. I have a mishmash of hardware, but all CPUs are
>>>>AMD
>>>> Phenoms.
>>>>
>>>> I used the convenience packages from
>>>> http://cloudstack.apt-get.eu/rhel/4.0 to do the upgrade.
>>>>
>>>> I have everything running, except for the system vms. Neither the
>>>> console proxy nor the secondary storage VMs are running, or even seem
>>>>to
>>>> try to start. I can start VMs from my templates, and everything
>>>>seems to
>>>> work fine, but (obviously) the console proxy doesn't work, and
>>>>machines
>>>> that require secondary storage won't work (but I don't currently have
>>>>any
>>>> of those).
>>>>
>>>> The related messages from the server:
>>>> 2012-11-29 09:44:56,261 TRACE [db.Transaction.Connection]
>>>> (consoleproxy-1:null) Creating a DB connection with no txn: for 0:
>>>> dbconn1086427264. Stack:
>>>>
>>>>-Transaction.prepareStatement:453-Transaction.prepareAutoCloseStatement
>>>>:4
>>>> 4
>>>>
>>>>6-GenericDaoBase.searchIncludingRemoved:380-DatabaseCallback.intercept:
>>>>34
>>>> -
>>>>
>>>>GenericDaoBase.searchIncludingRemoved:342-DatabaseCallback.intercept:34
>>>>-G
>>>> e
>>>>
>>>>nericDaoBase.listIncludingRemovedBy:894-GenericDaoBase.listBy:871-Gener
>>>>ic
>>>> D
>>>>
>>>>aoBase.listBy:884-DataCenterDaoImpl.listEnabledZones:366-DatabaseCallba
>>>>ck
>>>> .
>>>> intercept:34-ConsoleProxyManagerImpl.getScannablePools:1915
>>>> 2012-11-29 09:44:56,261 TRACE [db.Transaction.Statement]
>>>> (consoleproxy-1:null) Preparing: SELECT data_center.id,
>>>>data_center.name,
>>>> data_center.description, data_center.dns1, data_center.dns2,
>>>> data_center.internal_dns1, data_center.internal_dns2,
>>>> data_center.router_mac_address, data_center.guest_network_cidr,
>>>> data_center.domain_id, data_center.domain, data_center.networktype,
>>>> data_center.dns_provider, data_center.dhcp_provider,
>>>> data_center.gateway_provider, data_center.vpn_provider,
>>>> data_center.userdata_provider, data_center.lb_provider,
>>>> data_center.firewall_provider, data_center.mac_address,
>>>> data_center.zone_token, data_center.removed,
>>>> data_center.allocation_state, data_center.uuid,
>>>> data_center.is_security_group_enabled,
>>>> data_center.is_local_storage_enabled FROM data_center WHERE
>>>> data_center.allocation_state = ? AND data_center.removed IS NULL
>>>> 2012-11-29 09:44:56,264 TRACE [db.Transaction.Statement]
>>>> (consoleproxy-1:null) Closing:
>>>> org.apache.commons.dbcp.DelegatingPreparedStatement@63f5e4b6
>>>> 2012-11-29 09:44:56,264 TRACE [db.Transaction.Connection]
>>>> (consoleproxy-1:null) Closing DB connection: dbconn1086427264
>>>> 2012-11-29 09:44:56,264 DEBUG
>>>> [cloud.consoleproxy.ConsoleProxyManagerImpl] (consoleproxy-1:null)
>>>>Zone 1
>>>> is not ready to launch console proxy yet
>>>> 2012-11-29 09:44:56,265 TRACE [utils.db.GlobalLock]
>>>>(consoleproxy-1:null)
>>>> lock consoleproxy.scan.lock is returned to free state, total holding
>>>>time
>>>> :32
>>>> 2012-11-29 09:44:56,265 TRACE [utils.db.GlobalLock]
>>>>(consoleproxy-1:null)
>>>> lock consoleproxy.scan.lock is released, lock count :0
>>>> 2012-11-29 09:44:56,267 TRACE [utils.db.GenericDaoBase]
>>>> (consoleproxy-1:null) Select SQL: SELECT stack_maid.id,
>>>>stack_maid.msid,
>>>> stack_maid.thread_id, stack_maid.seq, stack_maid.cleanup_delegate,
>>>> stack_maid.cleanup_context, stack_maid.created FROM stack_maid WHERE
>>>> 2012-11-29 09:44:56,267 TRACE [utils.db.GenericDaoBase]
>>>> (consoleproxy-1:null) Remove SQL: No remove sql
>>>> 2012-11-29 09:44:56,267 TRACE [utils.db.GenericDaoBase]
>>>> (consoleproxy-1:null) Select by Id SQL: SELECT stack_maid.id,
>>>> stack_maid.msid, stack_maid.thread_id, stack_maid.seq,
>>>> stack_maid.cleanup_delegate, stack_maid.cleanup_context,
>>>> stack_maid.created FROM stack_maid WHERE stack_maid.id = ?
>>>>
>>>>
>>>> The systemVM was downloaded from
>>>>
>>>>http://download.cloud.com/templates/acton/acton-systemvm-02062012.qcow2
>>>>.b
>>>> z
>>>> 2 when my cloud was originally set up (which incidentally is the same
>>>>as
>>>> given in the installation instructions for 4.0-incubator).
>>>>
>>>> The system VM shows in the database as being DOWNLOADED:
>>>> id: 3
>>>> host_id: 2
>>>> template_id: 3
>>>> created: 2012-10-24 23:56:42
>>>> last_updated: 2012-11-27 12:04:05
>>>> job_id: NULL
>>>> download_pct: 100
>>>> size: 725811200
>>>> physical_size: 725811200
>>>> download_state: DOWNLOADED
>>>> error_str: NULL
>>>> local_path: NULL
>>>> install_path:
>>>> template/tmpl/1/3//b5d17e02-394a-4a01-bf13-6963b3127c39.qcow2
>>>> url:
>>>>
>>>>http://download.cloud.com/templates/acton/acton-systemvm-02062012.qcow2
>>>>.b
>>>> z
>>>> 2
>>>> destroyed: 0
>>>> is_copy: 0
>>>>
>>>>
>>>> My 3.0.2 CS installation ran perfectly since I set it up, but I'd
>>>>really
>>>> like to get 4.0-incubator up and running correctly.
>>>>
>>>> I've googled and tried pretty much everything I've found (which isn't
>>>> much), but to no avail. I can't find any reason that the management
>>>> server thinks the zone isn't ready to launch the system VMs. I've
>>>> restarted the pod, the cluster, the hosts in the cluster, the zone,
>>>>but I
>>>> still have the issue.
>>>>
>>>> Any help would be appreciated.
>>>>
>>>> Jeff
>>>
>>
>
Re: Zone 1 is not ready to launch console proxy yet
Posted by Jeff Reed <je...@jeffreed.net>.
Thanks Chiradeep for your help… here are the results from both queries:
mysql> SELECT data_center.id, data_center.name, data_center.description, data_center.dns1, data_center.dns2, data_center.internal_dns1, data_center.internal_dns2, data_center.router_mac_address, data_center.guest_network_cidr, data_center.domain_id, data_center.domain, data_center.networktype, data_center.dns_provider, data_center.dhcp_provider, data_center.gateway_provider, data_center.vpn_provider, data_center.userdata_provider, data_center.lb_provider, data_center.firewall_provider, data_center.mac_address, data_center.zone_token, data_center.removed, data_center.allocation_state, data_center.uuid, data_center.is_security_group_enabled, data_center.is_local_storage_enabled FROM data_center WHERE data_center.allocation_state = 'Enabled' AND data_center.removed IS NULL \G
*************************** 1. row ***************************
id: 1
name: zone
description: NULL
dns1: 192.168.1.1
dns2: NULL
internal_dns1: 192.168.1.1
internal_dns2: NULL
router_mac_address: 02:00:00:00:00:01
guest_network_cidr: NULL
domain_id: NULL
domain: NULL
networktype: Basic
dns_provider: VirtualRouter
dhcp_provider: VirtualRouter
gateway_provider: NULL
vpn_provider: NULL
userdata_provider: VirtualRouter
lb_provider: ElasticLoadBalancerVm
firewall_provider: NULL
mac_address: 117
zone_token: b862f66d-e1de-3992-bb5c-7d8bb9811769
removed: NULL
allocation_state: Enabled
uuid: 55450c92-4fdc-4933-a777-fb9b9f6ab243
is_security_group_enabled: 1
is_local_storage_enabled: 0
1 row in set (0.00 sec)
mysql> select * from (select h.data_center_id, h.type, count(*) as count from host as h INNER JOIN mshost as m ON h.mgmt_server_id=m.msid where h.status='Up' and h.type='SecondaryStorage' and m.last_update > '2012-12-04 12:39:26' group by h.data_center_id, h.type UNION ALL select h.data_center_id, h.type, count(*) as count from host as h INNER JOIN mshost as m ON h.mgmt_server_id=m.msid where h.status='Up' and h.type='Routing' and m.last_update > '2012-12-04 12:39:26' group by h.data_center_id, h.type) as t ORDER by t.data_center_id, t.type;
+----------------+---------+-------+
| data_center_id | type | count |
+----------------+---------+-------+
| 1 | Routing | 2 |
+----------------+---------+-------+
1 row in set (0.00 sec)
On Dec 3, 2012, at 10:12 PM, Chiradeep Vittal <Ch...@citrix.com> wrote:
> There's 3 minimum requirements to start a system vm:
> 1. The zone is enabled
> 2. There is primary storage ready that matches the type of the system vm
> offering (e.g., if only local is available, then system vm disk offering
> also needs to be local)
> 3. The system vm templates are DOWNLOADED
> 3. There are enough hosts in the 'UP' state
> select * from (select h.data_center_id, h.type, count(*) as count from
> host as h INNER JOIN mshost as m ON h.mgmt_server_id=m.msid
> where h.status='Up' and h.type='SecondaryStorage' and m.last_update > ?
> group by h.data_center_id, h.type UNION ALL select h.data_center_id,
> h.type, count(*) as count from host as h INNER JOIN mshost as m ON
> h.mgmt_server_id=m.msid where h.status='Up' and h.type='Routing' and
> m.last_update > ? group by h.data_center_id, h.type) as t ORDER by
> t.data_center_id, t.type;
>
> The '?' is to be replaced by a date equivalent to now() in GMT - 150 secs
>
>
>
> On 12/3/12 6:58 PM, "Chiradeep Vittal" <Ch...@citrix.com> wrote:
>
>> So what is the result of the below SQL query if you run it by hand?
>> Use data_center.allocation_state = 'Enabled'
>>
>>
>> On 11/29/12 9:54 AM, "Jeff Reed" <je...@jeffreed.net> wrote:
>>
>>> Hey folks,
>>>
>>> I'm having a problem after upgrading from cloudstack 3.0.2 to 4.0. I
>>> followed the instructions here:
>>> http://incubator.apache.org/cloudstack/docs/en-US/Apache_CloudStack/4.0.0
>>> -
>>> incubating/html/Release_Notes/upgrade-instructions.html#upgrade-from-3.0.
>>> 2
>>> -to-4.0
>>>
>>> I've upgraded the agent machines from CentOS 6.2 to CentOS 6.3 and all
>>> packages are current, and am running KVM as the hypervisor. The
>>> management server isn't running as an agent. There are two machines
>>> running as agents. I have a mishmash of hardware, but all CPUs are AMD
>>> Phenoms.
>>>
>>> I used the convenience packages from
>>> http://cloudstack.apt-get.eu/rhel/4.0 to do the upgrade.
>>>
>>> I have everything running, except for the system vms. Neither the
>>> console proxy nor the secondary storage VMs are running, or even seem to
>>> try to start. I can start VMs from my templates, and everything seems to
>>> work fine, but (obviously) the console proxy doesn't work, and machines
>>> that require secondary storage won't work (but I don't currently have any
>>> of those).
>>>
>>> The related messages from the server:
>>> 2012-11-29 09:44:56,261 TRACE [db.Transaction.Connection]
>>> (consoleproxy-1:null) Creating a DB connection with no txn: for 0:
>>> dbconn1086427264. Stack:
>>> -Transaction.prepareStatement:453-Transaction.prepareAutoCloseStatement:4
>>> 4
>>> 6-GenericDaoBase.searchIncludingRemoved:380-DatabaseCallback.intercept:34
>>> -
>>> GenericDaoBase.searchIncludingRemoved:342-DatabaseCallback.intercept:34-G
>>> e
>>> nericDaoBase.listIncludingRemovedBy:894-GenericDaoBase.listBy:871-Generic
>>> D
>>> aoBase.listBy:884-DataCenterDaoImpl.listEnabledZones:366-DatabaseCallback
>>> .
>>> intercept:34-ConsoleProxyManagerImpl.getScannablePools:1915
>>> 2012-11-29 09:44:56,261 TRACE [db.Transaction.Statement]
>>> (consoleproxy-1:null) Preparing: SELECT data_center.id, data_center.name,
>>> data_center.description, data_center.dns1, data_center.dns2,
>>> data_center.internal_dns1, data_center.internal_dns2,
>>> data_center.router_mac_address, data_center.guest_network_cidr,
>>> data_center.domain_id, data_center.domain, data_center.networktype,
>>> data_center.dns_provider, data_center.dhcp_provider,
>>> data_center.gateway_provider, data_center.vpn_provider,
>>> data_center.userdata_provider, data_center.lb_provider,
>>> data_center.firewall_provider, data_center.mac_address,
>>> data_center.zone_token, data_center.removed,
>>> data_center.allocation_state, data_center.uuid,
>>> data_center.is_security_group_enabled,
>>> data_center.is_local_storage_enabled FROM data_center WHERE
>>> data_center.allocation_state = ? AND data_center.removed IS NULL
>>> 2012-11-29 09:44:56,264 TRACE [db.Transaction.Statement]
>>> (consoleproxy-1:null) Closing:
>>> org.apache.commons.dbcp.DelegatingPreparedStatement@63f5e4b6
>>> 2012-11-29 09:44:56,264 TRACE [db.Transaction.Connection]
>>> (consoleproxy-1:null) Closing DB connection: dbconn1086427264
>>> 2012-11-29 09:44:56,264 DEBUG
>>> [cloud.consoleproxy.ConsoleProxyManagerImpl] (consoleproxy-1:null) Zone 1
>>> is not ready to launch console proxy yet
>>> 2012-11-29 09:44:56,265 TRACE [utils.db.GlobalLock] (consoleproxy-1:null)
>>> lock consoleproxy.scan.lock is returned to free state, total holding time
>>> :32
>>> 2012-11-29 09:44:56,265 TRACE [utils.db.GlobalLock] (consoleproxy-1:null)
>>> lock consoleproxy.scan.lock is released, lock count :0
>>> 2012-11-29 09:44:56,267 TRACE [utils.db.GenericDaoBase]
>>> (consoleproxy-1:null) Select SQL: SELECT stack_maid.id, stack_maid.msid,
>>> stack_maid.thread_id, stack_maid.seq, stack_maid.cleanup_delegate,
>>> stack_maid.cleanup_context, stack_maid.created FROM stack_maid WHERE
>>> 2012-11-29 09:44:56,267 TRACE [utils.db.GenericDaoBase]
>>> (consoleproxy-1:null) Remove SQL: No remove sql
>>> 2012-11-29 09:44:56,267 TRACE [utils.db.GenericDaoBase]
>>> (consoleproxy-1:null) Select by Id SQL: SELECT stack_maid.id,
>>> stack_maid.msid, stack_maid.thread_id, stack_maid.seq,
>>> stack_maid.cleanup_delegate, stack_maid.cleanup_context,
>>> stack_maid.created FROM stack_maid WHERE stack_maid.id = ?
>>>
>>>
>>> The systemVM was downloaded from
>>> http://download.cloud.com/templates/acton/acton-systemvm-02062012.qcow2.b
>>> z
>>> 2 when my cloud was originally set up (which incidentally is the same as
>>> given in the installation instructions for 4.0-incubator).
>>>
>>> The system VM shows in the database as being DOWNLOADED:
>>> id: 3
>>> host_id: 2
>>> template_id: 3
>>> created: 2012-10-24 23:56:42
>>> last_updated: 2012-11-27 12:04:05
>>> job_id: NULL
>>> download_pct: 100
>>> size: 725811200
>>> physical_size: 725811200
>>> download_state: DOWNLOADED
>>> error_str: NULL
>>> local_path: NULL
>>> install_path:
>>> template/tmpl/1/3//b5d17e02-394a-4a01-bf13-6963b3127c39.qcow2
>>> url:
>>> http://download.cloud.com/templates/acton/acton-systemvm-02062012.qcow2.b
>>> z
>>> 2
>>> destroyed: 0
>>> is_copy: 0
>>>
>>>
>>> My 3.0.2 CS installation ran perfectly since I set it up, but I'd really
>>> like to get 4.0-incubator up and running correctly.
>>>
>>> I've googled and tried pretty much everything I've found (which isn't
>>> much), but to no avail. I can't find any reason that the management
>>> server thinks the zone isn't ready to launch the system VMs. I've
>>> restarted the pod, the cluster, the hosts in the cluster, the zone, but I
>>> still have the issue.
>>>
>>> Any help would be appreciated.
>>>
>>> Jeff
>>
>
Re: Zone 1 is not ready to launch console proxy yet
Posted by Chiradeep Vittal <Ch...@citrix.com>.
There's 3 minimum requirements to start a system vm:
1. The zone is enabled
2. There is primary storage ready that matches the type of the system vm
offering (e.g., if only local is available, then system vm disk offering
also needs to be local)
3. The system vm templates are DOWNLOADED
3. There are enough hosts in the 'UP' state
select * from (select h.data_center_id, h.type, count(*) as count from
host as h INNER JOIN mshost as m ON h.mgmt_server_id=m.msid
where h.status='Up' and h.type='SecondaryStorage' and m.last_update > ?
group by h.data_center_id, h.type UNION ALL select h.data_center_id,
h.type, count(*) as count from host as h INNER JOIN mshost as m ON
h.mgmt_server_id=m.msid where h.status='Up' and h.type='Routing' and
m.last_update > ? group by h.data_center_id, h.type) as t ORDER by
t.data_center_id, t.type;
The '?' is to be replaced by a date equivalent to now() in GMT - 150 secs
On 12/3/12 6:58 PM, "Chiradeep Vittal" <Ch...@citrix.com> wrote:
>So what is the result of the below SQL query if you run it by hand?
>Use data_center.allocation_state = 'Enabled'
>
>
>On 11/29/12 9:54 AM, "Jeff Reed" <je...@jeffreed.net> wrote:
>
>>Hey folks,
>>
>>I'm having a problem after upgrading from cloudstack 3.0.2 to 4.0. I
>>followed the instructions here:
>>http://incubator.apache.org/cloudstack/docs/en-US/Apache_CloudStack/4.0.0
>>-
>>incubating/html/Release_Notes/upgrade-instructions.html#upgrade-from-3.0.
>>2
>>-to-4.0
>>
>>I've upgraded the agent machines from CentOS 6.2 to CentOS 6.3 and all
>>packages are current, and am running KVM as the hypervisor. The
>>management server isn't running as an agent. There are two machines
>>running as agents. I have a mishmash of hardware, but all CPUs are AMD
>>Phenoms.
>>
>>I used the convenience packages from
>>http://cloudstack.apt-get.eu/rhel/4.0 to do the upgrade.
>>
>>I have everything running, except for the system vms. Neither the
>>console proxy nor the secondary storage VMs are running, or even seem to
>>try to start. I can start VMs from my templates, and everything seems to
>>work fine, but (obviously) the console proxy doesn't work, and machines
>>that require secondary storage won't work (but I don't currently have any
>>of those).
>>
>>The related messages from the server:
>>2012-11-29 09:44:56,261 TRACE [db.Transaction.Connection]
>>(consoleproxy-1:null) Creating a DB connection with no txn: for 0:
>>dbconn1086427264. Stack:
>>-Transaction.prepareStatement:453-Transaction.prepareAutoCloseStatement:4
>>4
>>6-GenericDaoBase.searchIncludingRemoved:380-DatabaseCallback.intercept:34
>>-
>>GenericDaoBase.searchIncludingRemoved:342-DatabaseCallback.intercept:34-G
>>e
>>nericDaoBase.listIncludingRemovedBy:894-GenericDaoBase.listBy:871-Generic
>>D
>>aoBase.listBy:884-DataCenterDaoImpl.listEnabledZones:366-DatabaseCallback
>>.
>>intercept:34-ConsoleProxyManagerImpl.getScannablePools:1915
>>2012-11-29 09:44:56,261 TRACE [db.Transaction.Statement]
>>(consoleproxy-1:null) Preparing: SELECT data_center.id, data_center.name,
>>data_center.description, data_center.dns1, data_center.dns2,
>>data_center.internal_dns1, data_center.internal_dns2,
>>data_center.router_mac_address, data_center.guest_network_cidr,
>>data_center.domain_id, data_center.domain, data_center.networktype,
>>data_center.dns_provider, data_center.dhcp_provider,
>>data_center.gateway_provider, data_center.vpn_provider,
>>data_center.userdata_provider, data_center.lb_provider,
>>data_center.firewall_provider, data_center.mac_address,
>>data_center.zone_token, data_center.removed,
>>data_center.allocation_state, data_center.uuid,
>>data_center.is_security_group_enabled,
>>data_center.is_local_storage_enabled FROM data_center WHERE
>>data_center.allocation_state = ? AND data_center.removed IS NULL
>>2012-11-29 09:44:56,264 TRACE [db.Transaction.Statement]
>>(consoleproxy-1:null) Closing:
>>org.apache.commons.dbcp.DelegatingPreparedStatement@63f5e4b6
>>2012-11-29 09:44:56,264 TRACE [db.Transaction.Connection]
>>(consoleproxy-1:null) Closing DB connection: dbconn1086427264
>>2012-11-29 09:44:56,264 DEBUG
>>[cloud.consoleproxy.ConsoleProxyManagerImpl] (consoleproxy-1:null) Zone 1
>>is not ready to launch console proxy yet
>>2012-11-29 09:44:56,265 TRACE [utils.db.GlobalLock] (consoleproxy-1:null)
>>lock consoleproxy.scan.lock is returned to free state, total holding time
>>:32
>>2012-11-29 09:44:56,265 TRACE [utils.db.GlobalLock] (consoleproxy-1:null)
>>lock consoleproxy.scan.lock is released, lock count :0
>>2012-11-29 09:44:56,267 TRACE [utils.db.GenericDaoBase]
>>(consoleproxy-1:null) Select SQL: SELECT stack_maid.id, stack_maid.msid,
>>stack_maid.thread_id, stack_maid.seq, stack_maid.cleanup_delegate,
>>stack_maid.cleanup_context, stack_maid.created FROM stack_maid WHERE
>>2012-11-29 09:44:56,267 TRACE [utils.db.GenericDaoBase]
>>(consoleproxy-1:null) Remove SQL: No remove sql
>>2012-11-29 09:44:56,267 TRACE [utils.db.GenericDaoBase]
>>(consoleproxy-1:null) Select by Id SQL: SELECT stack_maid.id,
>>stack_maid.msid, stack_maid.thread_id, stack_maid.seq,
>>stack_maid.cleanup_delegate, stack_maid.cleanup_context,
>>stack_maid.created FROM stack_maid WHERE stack_maid.id = ?
>>
>>
>>The systemVM was downloaded from
>>http://download.cloud.com/templates/acton/acton-systemvm-02062012.qcow2.b
>>z
>>2 when my cloud was originally set up (which incidentally is the same as
>>given in the installation instructions for 4.0-incubator).
>>
>>The system VM shows in the database as being DOWNLOADED:
>> id: 3
>> host_id: 2
>> template_id: 3
>> created: 2012-10-24 23:56:42
>> last_updated: 2012-11-27 12:04:05
>> job_id: NULL
>> download_pct: 100
>> size: 725811200
>> physical_size: 725811200
>>download_state: DOWNLOADED
>> error_str: NULL
>> local_path: NULL
>> install_path:
>>template/tmpl/1/3//b5d17e02-394a-4a01-bf13-6963b3127c39.qcow2
>> url:
>>http://download.cloud.com/templates/acton/acton-systemvm-02062012.qcow2.b
>>z
>>2
>> destroyed: 0
>> is_copy: 0
>>
>>
>>My 3.0.2 CS installation ran perfectly since I set it up, but I'd really
>>like to get 4.0-incubator up and running correctly.
>>
>>I've googled and tried pretty much everything I've found (which isn't
>>much), but to no avail. I can't find any reason that the management
>>server thinks the zone isn't ready to launch the system VMs. I've
>>restarted the pod, the cluster, the hosts in the cluster, the zone, but I
>>still have the issue.
>>
>>Any help would be appreciated.
>>
>>Jeff
>
Re: Zone 1 is not ready to launch console proxy yet
Posted by Chiradeep Vittal <Ch...@citrix.com>.
So what is the result of the below SQL query if you run it by hand?
Use data_center.allocation_state = 'Enabled'
On 11/29/12 9:54 AM, "Jeff Reed" <je...@jeffreed.net> wrote:
>Hey folks,
>
>I'm having a problem after upgrading from cloudstack 3.0.2 to 4.0. I
>followed the instructions here:
>http://incubator.apache.org/cloudstack/docs/en-US/Apache_CloudStack/4.0.0-
>incubating/html/Release_Notes/upgrade-instructions.html#upgrade-from-3.0.2
>-to-4.0
>
>I've upgraded the agent machines from CentOS 6.2 to CentOS 6.3 and all
>packages are current, and am running KVM as the hypervisor. The
>management server isn't running as an agent. There are two machines
>running as agents. I have a mishmash of hardware, but all CPUs are AMD
>Phenoms.
>
>I used the convenience packages from
>http://cloudstack.apt-get.eu/rhel/4.0 to do the upgrade.
>
>I have everything running, except for the system vms. Neither the
>console proxy nor the secondary storage VMs are running, or even seem to
>try to start. I can start VMs from my templates, and everything seems to
>work fine, but (obviously) the console proxy doesn't work, and machines
>that require secondary storage won't work (but I don't currently have any
>of those).
>
>The related messages from the server:
>2012-11-29 09:44:56,261 TRACE [db.Transaction.Connection]
>(consoleproxy-1:null) Creating a DB connection with no txn: for 0:
>dbconn1086427264. Stack:
>-Transaction.prepareStatement:453-Transaction.prepareAutoCloseStatement:44
>6-GenericDaoBase.searchIncludingRemoved:380-DatabaseCallback.intercept:34-
>GenericDaoBase.searchIncludingRemoved:342-DatabaseCallback.intercept:34-Ge
>nericDaoBase.listIncludingRemovedBy:894-GenericDaoBase.listBy:871-GenericD
>aoBase.listBy:884-DataCenterDaoImpl.listEnabledZones:366-DatabaseCallback.
>intercept:34-ConsoleProxyManagerImpl.getScannablePools:1915
>2012-11-29 09:44:56,261 TRACE [db.Transaction.Statement]
>(consoleproxy-1:null) Preparing: SELECT data_center.id, data_center.name,
>data_center.description, data_center.dns1, data_center.dns2,
>data_center.internal_dns1, data_center.internal_dns2,
>data_center.router_mac_address, data_center.guest_network_cidr,
>data_center.domain_id, data_center.domain, data_center.networktype,
>data_center.dns_provider, data_center.dhcp_provider,
>data_center.gateway_provider, data_center.vpn_provider,
>data_center.userdata_provider, data_center.lb_provider,
>data_center.firewall_provider, data_center.mac_address,
>data_center.zone_token, data_center.removed,
>data_center.allocation_state, data_center.uuid,
>data_center.is_security_group_enabled,
>data_center.is_local_storage_enabled FROM data_center WHERE
>data_center.allocation_state = ? AND data_center.removed IS NULL
>2012-11-29 09:44:56,264 TRACE [db.Transaction.Statement]
>(consoleproxy-1:null) Closing:
>org.apache.commons.dbcp.DelegatingPreparedStatement@63f5e4b6
>2012-11-29 09:44:56,264 TRACE [db.Transaction.Connection]
>(consoleproxy-1:null) Closing DB connection: dbconn1086427264
>2012-11-29 09:44:56,264 DEBUG
>[cloud.consoleproxy.ConsoleProxyManagerImpl] (consoleproxy-1:null) Zone 1
>is not ready to launch console proxy yet
>2012-11-29 09:44:56,265 TRACE [utils.db.GlobalLock] (consoleproxy-1:null)
>lock consoleproxy.scan.lock is returned to free state, total holding time
>:32
>2012-11-29 09:44:56,265 TRACE [utils.db.GlobalLock] (consoleproxy-1:null)
>lock consoleproxy.scan.lock is released, lock count :0
>2012-11-29 09:44:56,267 TRACE [utils.db.GenericDaoBase]
>(consoleproxy-1:null) Select SQL: SELECT stack_maid.id, stack_maid.msid,
>stack_maid.thread_id, stack_maid.seq, stack_maid.cleanup_delegate,
>stack_maid.cleanup_context, stack_maid.created FROM stack_maid WHERE
>2012-11-29 09:44:56,267 TRACE [utils.db.GenericDaoBase]
>(consoleproxy-1:null) Remove SQL: No remove sql
>2012-11-29 09:44:56,267 TRACE [utils.db.GenericDaoBase]
>(consoleproxy-1:null) Select by Id SQL: SELECT stack_maid.id,
>stack_maid.msid, stack_maid.thread_id, stack_maid.seq,
>stack_maid.cleanup_delegate, stack_maid.cleanup_context,
>stack_maid.created FROM stack_maid WHERE stack_maid.id = ?
>
>
>The systemVM was downloaded from
>http://download.cloud.com/templates/acton/acton-systemvm-02062012.qcow2.bz
>2 when my cloud was originally set up (which incidentally is the same as
>given in the installation instructions for 4.0-incubator).
>
>The system VM shows in the database as being DOWNLOADED:
> id: 3
> host_id: 2
> template_id: 3
> created: 2012-10-24 23:56:42
> last_updated: 2012-11-27 12:04:05
> job_id: NULL
> download_pct: 100
> size: 725811200
> physical_size: 725811200
>download_state: DOWNLOADED
> error_str: NULL
> local_path: NULL
> install_path:
>template/tmpl/1/3//b5d17e02-394a-4a01-bf13-6963b3127c39.qcow2
> url:
>http://download.cloud.com/templates/acton/acton-systemvm-02062012.qcow2.bz
>2
> destroyed: 0
> is_copy: 0
>
>
>My 3.0.2 CS installation ran perfectly since I set it up, but I'd really
>like to get 4.0-incubator up and running correctly.
>
>I've googled and tried pretty much everything I've found (which isn't
>much), but to no avail. I can't find any reason that the management
>server thinks the zone isn't ready to launch the system VMs. I've
>restarted the pod, the cluster, the hosts in the cluster, the zone, but I
>still have the issue.
>
>Any help would be appreciated.
>
>Jeff