[jira] [Created] (CLOUDSTACK-5382) Random failure when trying to deploy an instance from a template which is available across zones

["Kirk Kosinski (JIRA)" <ji...@apache.org>]

Kirk Kosinski created CLOUDSTACK-5382:
-----------------------------------------

             Summary: Random failure when trying to deploy an instance from a template which is available across zones
                 Key: CLOUDSTACK-5382
                 URL: https://issues.apache.org/jira/browse/CLOUDSTACK-5382
             Project: CloudStack
          Issue Type: Bug
      Security Level: Public (Anyone can view this level - this is the default.)
          Components: Management Server
    Affects Versions: 4.2.1
            Reporter: Kirk Kosinski


When trying to deploy an instance from a template which is available across zones, CloudStack randomly selects the wrong secondary storage. Selection happens by a random method from the available secondary storages which belong to multiple zones.

2013-11-13 10:29:05,402 TRACE [db.Transaction.Statement] (Job-Executor-102:job-2102 = [ 29ea7d9b-ddc6-49eb-af54-689d5c002a8d ]) Closing: com.mysql.jdbc.JDBC4PreparedStatement@5080267f: SELECT template_store_ref.id, template_store_ref.store_id, template_store_ref.template_id, template_store_ref.store_role, template_store_ref.created, template_store_ref.last_updated, template_store_ref.download_pct, template_store_ref.size, template_store_ref.physical_size, template_store_ref.download_state, template_store_ref.local_path, template_store_ref.error_str, template_store_ref.job_id, template_store_ref.install_path, template_store_ref.url, template_store_ref.download_url, template_store_ref.download_url_created, template_store_ref.is_copy, template_store_ref.destroyed, template_store_ref.update_count, template_store_ref.updated, template_store_ref.state, template_store_ref.ref_cnt FROM template_store_ref WHERE template_store_ref.template_id = 211 AND template_store_ref.store_role = 'Image' AND template_store_ref.destroyed = 0 ORDER BY RAND() LIMIT 1

For example, for template 211, attempts were made to access it from the wrong secondary storage share three times before the correct secondary storage was used. The issue is a DB query with "WHERE template_store_ref.template_id = 211 AND template_store_ref.store_role = 'Image' AND template_store_ref.destroyed = 0 ORDER BY RAND() LIMIT 1" This results in a random image_store.id but not necessarily one in the correct zone.

20:22:45,183 INFO VmwareStorageProcessor:217 - Template 68dd65a2-ab83-3c49-8b9d-b54cd972c88e is not setup yet, setup template from secondary storage with uuid name: 46f1e17908053d9a8615e5c64ecc3811
20:22:45,214 INFO VmwareStorageProcessor:130 - Executing copyTemplateFromSecondaryToPrimary. secondaryStorage: nfs://nfshost/path/to/share, templatePathAtSecondaryStorage: template/tmpl/9/211/, templateName: 68dd65a2-ab83-3c49-8b9d-b54cd972c88e
20:22:45,535 WARN NfsSecondaryStorageResource:2225 - Unable to mount 10.x.x.x:/path/to/share due to mount.nfs: access denied by server while mounting 10.x.x.x:/path/to/share
20:22:45,535 INFO VmwareStorageProcessor:135 - Secondary storage mount point: /mnt/SecStorage/6cd05351-e52d-3c98-8cd8-ace877ed7518
20:22:45,535 INFO VmwareStorageProcessor:147 - Executing command: tar --no-same-owner -xf /mnt/SecStorage/6cd05351-e52d-3c98-8cd8-ace877ed7518/template/tmpl/9/211//68dd65a2-ab83-3c49-8b9d-b54cd972c88e.ova
20:22:45,541 WARN VmwareStorageProcessor:267 - Exception: tar --no-same-owner -xf /mnt/SecStorage/6cd05351-e52d-3c98-8cd8-ace877ed7518/template/tmpl/9/211//68dd65a2-ab83-3c49-8b9d-b54cd972c88e.ova
java.io.IOException: Cannot run program "tar" (in directory "/mnt/SecStorage/6cd05351-e52d-3c98-8cd8-ace877ed7518/template/tmpl/9/211"): java.io.IOException: error=2, No such file or directory 



--
This message was sent by Atlassian JIRA
(v6.1#6144)