You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@cloudstack.apache.org by "Min Chen (JIRA)" <ji...@apache.org> on 2013/07/25 22:57:49 UTC
[jira] [Resolved] (CLOUDSTACK-3667) [Object_Store_Refactor][VMWare]
System VMs StartCommand failed due to Exception: java.lang.Exception
Message: secondary storage for dc 3 is not ready yet?
[ https://issues.apache.org/jira/browse/CLOUDSTACK-3667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Min Chen resolved CLOUDSTACK-3667.
----------------------------------
Resolution: Fixed
> [Object_Store_Refactor][VMWare] System VMs StartCommand failed due to Exception: java.lang.Exception Message: secondary storage for dc 3 is not ready yet?
> ----------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: CLOUDSTACK-3667
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-3667
> Project: CloudStack
> Issue Type: Bug
> Security Level: Public(Anyone can view this level - this is the default.)
> Components: Management Server, VMware
> Affects Versions: 4.2.0
> Environment: Latest build from ACS 4.2 Branch.
> Zone: Advanced with VMWare cluster
> Storage: S3 and ISCSI(Primary)
> Reporter: Sanjeev N
> Assignee: Min Chen
> Priority: Blocker
> Fix For: 4.2.0
>
> Attachments: management-server.rar
>
>
> [Object_Store_Refactor][VMWare] System VMs StartCommand failed due to Exception: java.lang.Exception Message: secondary storage for dc 3 is not ready yet?
> System VMs failed to start in case of VMWare cluster due to NULL in URL for S3 image store in image_store table with role Image.
> I looked at the following code in incubator-cloudstack/plugins/hypervisors/vmware/src/com/cloud/hypervisor/vmware/resource/VmwareResource.java :
> // prepare systemvm patch ISO
> if (vmSpec.getType() != VirtualMachine.Type.User) {
> // attach ISO (for patching of system VM)
> String secStoreUrl = mgr.getSecondaryStorageStoreUrl(Long.parseLong(_dcId));
> if(secStoreUrl == null) {
> String msg = "secondary storage for dc " + _dcId + " is not ready yet?";
> throw new Exception(msg);
> }
> mgr.prepareSecondaryStorageStore(secStoreUrl);
> Since secStoreUrl is matching with null , system vms starting failing with following exception message:
> java.lang.Exception: secondary storage for dc 3 is not ready yet?
> Steps to Reproduce:
> ================
> 1.Bring up CS in advanced zone with vmware cluster
> 2.Add S3 image store and NFS cache store in the zone
> 3.Enable the zone
> Observations:
> ===========
> 2013-07-19 11:50:10,861 WARN [vmware.resource.VmwareResource] (DirectAgent-97:10.147.40.13) StartCommand failed due to Exception: java.lang.Exception
> Message: secondary storage for dc 3 is not ready yet?
> java.lang.Exception: secondary storage for dc 3 is not ready yet?
> at com.cloud.hypervisor.vmware.resource.VmwareResource.execute(VmwareResource.java:2628)
> at com.cloud.hypervisor.vmware.resource.VmwareResource.executeRequest(VmwareResource.java:519)
> at com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:186)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:679)
> Enabled TRACE in log4j-cloud.xml and found the following sql statements executed before the exception:
> mysql> SELECT image_store.id, image_store.name, image_store.uuid, image_store.protocol, image_store.url, image_store.image_provider_name, image_store.data_center_id, image_store.scope, image_store.created, image_store.removed, image_store.role, image_store.parent, image_store.total_size, image_store.used_bytes FROM image_store WHERE image_store.role = 'Image' AND (image_store.scope = 'REGION' OR image_store.data_center_id = 3 ) AND image_store.removed IS NULL;
> +----+------------+--------------------------------------+----------+------+---------------------+----------------+--------+---------------------+---------+-------+--------+------------+------------+
> | id | name | uuid | protocol | url | image_provider_name | data_center_id | scope | created | removed | role | parent | total_size | used_bytes |
> +----+------------+--------------------------------------+----------+------+---------------------+----------------+--------+---------------------+---------+-------+--------+------------+------------+
> | 8 | obj_store | ada9e8f2-2bf1-46b7-b19d-a9bd46a0a29d | http | NULL | S3 | NULL | REGION | 2013-07-16 12:07:09 | NULL | Image | NULL | NULL | NULL |
> | 17 | obj_store2 | 9e55b78b-cead-45e4-a84e-b35896d961a8 | http | NULL | S3 | NULL | REGION | 2013-07-19 15:12:45 | NULL | Image | NULL | NULL | NULL |
> +----+------------+--------------------------------------+----------+------+---------------------+----------------+--------+---------------------+---------+-------+--------+------------+------------+
> 2 rows in set (0.00 sec)
> mysql> SELECT image_store.id, image_store.name, image_store.uuid, image_store.protocol, image_store.url, image_store.image_provider_name, image_store.data_center_id, image_store.scope, image_store.created, image_store.removed, image_store.role, image_store.parent, image_store.total_size, image_store.used_bytes FROM image_store WHERE image_store.id = 17 AND image_store.removed IS NULL;
> +----+------------+--------------------------------------+----------+------+---------------------+----------------+--------+---------------------+---------+-------+--------+------------+------------+
> | id | name | uuid | protocol | url | image_provider_name | data_center_id | scope | created | removed | role | parent | total_size | used_bytes |
> +----+------------+--------------------------------------+----------+------+---------------------+----------------+--------+---------------------+---------+-------+--------+------------+------------+
> | 17 | obj_store2 | 9e55b78b-cead-45e4-a84e-b35896d961a8 | http | NULL | S3 | NULL | REGION | 2013-07-19 15:12:45 | NULL | Image | NULL | NULL | NULL |
> +----+------------+--------------------------------------+----------+------+---------------------+----------------+--------+---------------------+---------+-------+--------+------------+------------+
> 1 row in set (0.00 sec)
> mysql> SELECT image_store.id, image_store.name, image_store.uuid, image_store.protocol, image_store.url, image_store.image_provider_name, image_store.data_center_id, image_store.scope, image_store.created, image_store.removed, image_store.role, image_store.parent, image_store.total_size, image_store.used_bytes FROM image_store WHERE image_store.id = 8 AND image_store.removed IS NULL;
> +----+-----------+--------------------------------------+----------+------+---------------------+----------------+--------+---------------------+---------+-------+--------+------------+------------+
> | id | name | uuid | protocol | url | image_provider_name | data_center_id | scope | created | removed | role | parent | total_size | used_bytes |
> +----+-----------+--------------------------------------+----------+------+---------------------+----------------+--------+---------------------+---------+-------+--------+------------+------------+
> | 8 | obj_store | ada9e8f2-2bf1-46b7-b19d-a9bd46a0a29d | http | NULL | S3 | NULL | REGION | 2013-07-16 12:07:09 | NULL | Image | NULL | NULL | NULL |
> +----+-----------+--------------------------------------+----------+------+---------------------+----------------+--------+---------------------+---------+-------+--------+------------+------------+
> 1 row in set (0.00 sec)
> ulr value is set to null incase of S3 storage provider.
> Following is the image_store table content:
> =================================
> mysql> select * from image_store where removed is null\G;
> *************************** 1. row ***************************
> id: 7
> name: nfs://10.147.28.7/export/home/sanjeev/sec_kvm_os
> image_provider_name: NFS
> protocol: nfs
> url: nfs://10.147.28.7/export/home/sanjeev/sec_kvm_os
> data_center_id: 1
> scope: ZONE
> role: ImageCache
> uuid: aa07c444-910a-43ff-bfb7-621c0ddc6223
> parent: NULL
> created: 2013-07-16 12:07:09
> removed: NULL
> total_size: NULL
> used_bytes: NULL
> *************************** 2. row ***************************
> id: 8
> name: obj_store
> image_provider_name: S3
> protocol: http
> url: NULL
> data_center_id: NULL
> scope: REGION
> role: Image
> uuid: ada9e8f2-2bf1-46b7-b19d-a9bd46a0a29d
> parent: NULL
> created: 2013-07-16 12:07:09
> removed: NULL
> total_size: NULL
> used_bytes: NULL
> *************************** 3. row ***************************
> id: 16
> name: nfs://10.147.28.7/export/home/sanjeev/sec_esx_os
> image_provider_name: NFS
> protocol: nfs
> url: nfs://10.147.28.7/export/home/sanjeev/sec_esx_os
> data_center_id: 3
> scope: ZONE
> role: ImageCache
> uuid: c3e72ffb-2040-4110-856a-ffabf4ca5992
> parent: NULL
> created: 2013-07-19 15:12:45
> removed: NULL
> total_size: NULL
> used_bytes: NULL
> *************************** 4. row ***************************
> id: 17
> name: obj_store2
> image_provider_name: S3
> protocol: http
> url: NULL
> data_center_id: NULL
> scope: REGION
> role: Image
> uuid: 9e55b78b-cead-45e4-a84e-b35896d961a8
> parent: NULL
> created: 2013-07-19 15:12:45
> removed: NULL
> total_size: NULL
> used_bytes: NULL
> 4 rows in set (0.00 sec)
> ERROR:
> No query specified
> Attached management server log file.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira