Can't create proper template from VM on S3 secondary storage environment

[Akihiko Ota <ak...@aist.go.jp>]

Hello,

I am running CloudStack 4.3.0 on CentOS 6.5. My environment is
- KVM Hypervisor.
- using S3 provider to socondary storage. The storage system is using RADOS.
- using local storage to primary storage.

When I create more than one templates from an existing virtual
machine, the second and the subsequent template is created the same as
first one. It doesn't occur in the case of using NFS secondary
storage. So I guess this is the S3 provider specific issue.
This could reproduce as following procedure.

(1) create a virtual machine (name "foo" tentatively), and modify some files on ROOT volume of "foo".
(2) stop "foo". and create template "template-foo.1" from ROOT volume of "foo".
(3) reboot (not re-create) "foo", and modify some files on disk again.
(4) stop "foo". and create template "template-foo.2" from ROOT volume of "foo".

I expect that template-foo.2 keeps (3)'s changes. But template-foo.2
is the same as template-foo.1. (3)'s changes are not recorded.

As of (4), CloudStack outputs the following DEBUG message to
cloudstack-management.log.

2014-08-07 19:23:20,094 DEBUG [o.a.c.s.c.m.StorageCacheManagerImpl] (Job-Executor-58:ctx-b00fbe22 ctx-6339bae6) there is already one in the cache store

When creating templates on NFS secondary storage environment, this
message is not output.

This issue also exists on CloudStack 4.2.1. Any ideas on how to avoid
this issue?

Thanks,

-- 
Akihiko Ota