Where should a system VM image be uncompressed?

[Donal Lafferty <do...@citrix.com>]

I noticed that the system VM template is stored in S3 as a .bz2.  E.g. as a .vhd.bz2 when using a Hyper-V hypervisor.

Naturally, you can't run a bz2.  Nor can you make a thin copy of it, say if it's a downloaded as a TEMPLATE to a primary storage pool.

Should it be uncompressed before it goes into S3, when it is copied from S3 to primary storage, or when a volume is created from the TEMPLATE?

RE: Where should a system VM image be uncompressed?

[Donal Lafferty <do...@citrix.com>]

> -----Original Message-----
> From: Edison Su [mailto:Edison.su@citrix.com]
> Sent: 27 June 2013 1:38 AM
> To: dev@cloudstack.apache.org
> Subject: RE: Where should a system VM image be uncompressed?
> 
> 
> 
> > -----Original Message-----
> > From: Donal Lafferty [mailto:donal.lafferty@citrix.com]
> > Sent: Wednesday, June 26, 2013 10:01 AM
> > To: dev@cloudstack.apache.org
> > Subject: Where should a system VM image be uncompressed?
> >
> > I noticed that the system VM template is stored in S3 as a .bz2.  E.g.
> > as a .vhd.bz2 when using a Hyper-V hypervisor.
> >
> > Naturally, you can't run a bz2.  Nor can you make a thin copy of it,
> > say if it's a downloaded as a TEMPLATE to a primary storage pool.
> >
> > Should it be uncompressed before it goes into S3, when it is copied
> > from S3 to primary storage, or when a volume is created from the
> TEMPLATE?
> There are two options:
> 1. Put an uncompressed template url into cloudstack db for HyperV system
> vm template, and disable register compressed template/ISO into S3, if the
> zone is for hyperV. If there is no staging area between S3 and primary
> storage, there is no place to unzip template.
> 2. Add a file system supported by hyperV as staging area, so that we can copy
> compressed template from S3 into that staging area, then unzip, and import
> into hyperV. Currently, we can add NFS as staging area, possible you can add
> CIFS as staging area also.

 [Donal Lafferty] 
I would prefer to update the database for now, and introduce a staging area in the future.

There are NuGet packages from the authors of bzip2 and zlib that would allow an agent to uncompress files on a CIFS share provided their licensing terms are acceptable.

DL

RE: Where should a system VM image be uncompressed?

[Edison Su <Ed...@citrix.com>]

> -----Original Message-----
> From: Donal Lafferty [mailto:donal.lafferty@citrix.com]
> Sent: Wednesday, June 26, 2013 10:01 AM
> To: dev@cloudstack.apache.org
> Subject: Where should a system VM image be uncompressed?
> 
> I noticed that the system VM template is stored in S3 as a .bz2.  E.g. as
> a .vhd.bz2 when using a Hyper-V hypervisor.
> 
> Naturally, you can't run a bz2.  Nor can you make a thin copy of it, say if it's a
> downloaded as a TEMPLATE to a primary storage pool.
> 
> Should it be uncompressed before it goes into S3, when it is copied from S3
> to primary storage, or when a volume is created from the TEMPLATE?
There are two options:
1. Put an uncompressed template url into cloudstack db for HyperV system vm template, and disable register compressed template/ISO into S3, if the zone is for hyperV. If there is no staging area between S3 and primary storage, there is no place to unzip template.
2. Add a file system supported by hyperV as staging area, so that we can copy compressed template from S3 into that staging area, then unzip, and import into hyperV. Currently, we can add NFS as staging area, possible you can add CIFS as staging area also.

Re: Where should a system VM image be uncompressed?

[Min Chen <mi...@citrix.com>]

That is expected behavior. We will store whatever format provided by user
in registering a template to S3. Currently uncompressing is done when the
template is copied from s3 to primary storage if NFS staging is enabled.
In your case, since you are not using NFS staging, some code has to be
added to uncompress when it is copied from S3 to primary storage when NFS
caching is disabled.

Thanks
-min

On 6/26/13 10:00 AM, "Donal Lafferty" <do...@citrix.com> wrote:

>I noticed that the system VM template is stored in S3 as a .bz2.  E.g. as
>a .vhd.bz2 when using a Hyper-V hypervisor.
>
>Naturally, you can't run a bz2.  Nor can you make a thin copy of it, say
>if it's a downloaded as a TEMPLATE to a primary storage pool.
>
>Should it be uncompressed before it goes into S3, when it is copied from
>S3 to primary storage, or when a volume is created from the TEMPLATE?
>