secondary storage questions

[Marcus Sorensen <sh...@gmail.com>]

Sort of embarrassed to ask this, but I'm unclear on a few of the
aspects of secondary storage.

1) I can see that if I add multiple secondary storage locations, they
each get a copy of the registered templates. Are these copied from one
secondary storage to another, or from the url of the registered
template?

2) It seems that the intent is to mirror templates across all
secondary storages, zone-wide, correct? But no copies are made for the
snapshots or volumes dirs. This may point to the idea that templates
are always downloaded via URLs

3) It sounds like the total size of secondary storage is somewhat
limited by templates, e.g. if all of you secondary storage is in
templates, adding another secondary storage server won't increase my
capacity, correct?

Re: secondary storage questions

[Nitin Mehta <Ni...@citrix.com>]

On 19/07/13 8:47 AM, "Mike Tutkowski" <mi...@solidfire.com> wrote:

>When we use the word "template" here, do we mean both templates (like
>OVAs)
>and ISOs?

Yes, that¹s correct.

>
>Thanks!
>
>
>On Wed, Jun 5, 2013 at 5:07 PM, Marcus Sorensen <sh...@gmail.com>
>wrote:
>
>> Thanks, I have one follow-up:
>>
>> On Tue, Jun 4, 2013 at 12:10 AM, Nitin Mehta <Ni...@citrix.com>
>> wrote:
>> > All the public templates will be replicated to all the secondary
>>storage
>> > for redundancy but private templates and uploaded volumes should be
>>kept
>> > in one of the randomly chosen secondary storage. Snapshots will be
>> > randomly copied to one of the secondary storage (the chain of
>>incremental
>> > snapshots are kept on the same secondary storage.). Currently these
>> > algorithms are hardcoded and so in case you need flexibility please
>>open
>> > an enhancement for the same.
>> >
>> >
>> > On 04/06/13 2:57 AM, "Marcus Sorensen" <sh...@gmail.com> wrote:
>> >
>> >>Anyone?
>> >>
>> >>We are also noticing that if we register a template, that template is
>> >>registered and ready, but becomes unusable if the URL for the template
>> >>stops working at any time. Seems like we attempt to red download it
>>every
>> >>so often. So no one-time registration. Is this by design?
>> >
>> > Whenever there is a handshake between MS and the agent running on the
>> SSVM
>> > there is a template sync which syncs the template status on the MS DB
>>and
>> > the template's physical Location and triggers the download if its not
>> > complete or retries in case there was some error. (This handshake
>>happens
>> > on ssvm re/start, agent on SSVM re/start and MS re/start).
>>
>> This implies that once the template is downloaded and is ready, the
>> URL doesn't need to stick around. We have some instances where we may
>> be downloading templates via one-time URLs. We had seen some things
>> that made us think that the template URLs are checked and possibly
>> redownloaded on occasion, so I wanted to clarify this, perhaps our
>> interpretations were wrong. Obviously if the URLs are checked and a
>> redownload is attempted for any reason, it will fail.
>>
>> >
>> >
>> >>On May 31, 2013 2:35 PM, "Marcus Sorensen" <sh...@gmail.com>
>>wrote:
>> >>
>> >>> Sort of embarrassed to ask this, but I'm unclear on a few of the
>> >>> aspects of secondary storage.
>> >>>
>> >>> 1) I can see that if I add multiple secondary storage locations,
>>they
>> >>> each get a copy of the registered templates. Are these copied from
>>one
>> >>> secondary storage to another, or from the url of the registered
>> >>> template?
>> >
>> > From the url of the template. Should happen only for public
>>templates. If
>> > you see otherwise please file a bug.
>> >
>> >>>
>> >>> 2) It seems that the intent is to mirror templates across all
>> >>> secondary storages, zone-wide, correct? But no copies are made for
>>the
>> >>> snapshots or volumes dirs. This may point to the idea that templates
>> >>> are always downloaded via URLs
>> >
>> > Multiple sec. storage are added for adding more storage and making
>>public
>> > templates highly available.
>> > Templates are downloaded via URLs but can also be created from
>>snapshots
>> > and volumes
>> >
>> >>>
>> >>> 3) It sounds like the total size of secondary storage is somewhat
>> >>> limited by templates, e.g. if all of you secondary storage is in
>> >>> templates, adding another secondary storage server won't increase my
>> >>> capacity, correct?
>> >
>> > Not correct, as per explanation above.
>> >
>> >>>
>> >
>>
>
>
>
>-- 
>*Mike Tutkowski*
>*Senior CloudStack Developer, SolidFire Inc.*
>e: mike.tutkowski@solidfire.com
>o: 303.746.7302
>Advancing the way the world uses the
>cloud<http://solidfire.com/solution/overview/?video=play>
>**

Re: secondary storage questions

[Mike Tutkowski <mi...@solidfire.com>]

When we use the word "template" here, do we mean both templates (like OVAs)
and ISOs?

Thanks!


On Wed, Jun 5, 2013 at 5:07 PM, Marcus Sorensen <sh...@gmail.com> wrote:

> Thanks, I have one follow-up:
>
> On Tue, Jun 4, 2013 at 12:10 AM, Nitin Mehta <Ni...@citrix.com>
> wrote:
> > All the public templates will be replicated to all the secondary storage
> > for redundancy but private templates and uploaded volumes should be kept
> > in one of the randomly chosen secondary storage. Snapshots will be
> > randomly copied to one of the secondary storage (the chain of incremental
> > snapshots are kept on the same secondary storage.). Currently these
> > algorithms are hardcoded and so in case you need flexibility please open
> > an enhancement for the same.
> >
> >
> > On 04/06/13 2:57 AM, "Marcus Sorensen" <sh...@gmail.com> wrote:
> >
> >>Anyone?
> >>
> >>We are also noticing that if we register a template, that template is
> >>registered and ready, but becomes unusable if the URL for the template
> >>stops working at any time. Seems like we attempt to red download it every
> >>so often. So no one-time registration. Is this by design?
> >
> > Whenever there is a handshake between MS and the agent running on the
> SSVM
> > there is a template sync which syncs the template status on the MS DB and
> > the template's physical Location and triggers the download if its not
> > complete or retries in case there was some error. (This handshake happens
> > on ssvm re/start, agent on SSVM re/start and MS re/start).
>
> This implies that once the template is downloaded and is ready, the
> URL doesn't need to stick around. We have some instances where we may
> be downloading templates via one-time URLs. We had seen some things
> that made us think that the template URLs are checked and possibly
> redownloaded on occasion, so I wanted to clarify this, perhaps our
> interpretations were wrong. Obviously if the URLs are checked and a
> redownload is attempted for any reason, it will fail.
>
> >
> >
> >>On May 31, 2013 2:35 PM, "Marcus Sorensen" <sh...@gmail.com> wrote:
> >>
> >>> Sort of embarrassed to ask this, but I'm unclear on a few of the
> >>> aspects of secondary storage.
> >>>
> >>> 1) I can see that if I add multiple secondary storage locations, they
> >>> each get a copy of the registered templates. Are these copied from one
> >>> secondary storage to another, or from the url of the registered
> >>> template?
> >
> > From the url of the template. Should happen only for public templates. If
> > you see otherwise please file a bug.
> >
> >>>
> >>> 2) It seems that the intent is to mirror templates across all
> >>> secondary storages, zone-wide, correct? But no copies are made for the
> >>> snapshots or volumes dirs. This may point to the idea that templates
> >>> are always downloaded via URLs
> >
> > Multiple sec. storage are added for adding more storage and making public
> > templates highly available.
> > Templates are downloaded via URLs but can also be created from snapshots
> > and volumes
> >
> >>>
> >>> 3) It sounds like the total size of secondary storage is somewhat
> >>> limited by templates, e.g. if all of you secondary storage is in
> >>> templates, adding another secondary storage server won't increase my
> >>> capacity, correct?
> >
> > Not correct, as per explanation above.
> >
> >>>
> >
>



-- 
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*™*

Re: secondary storage questions

[Marcus Sorensen <sh...@gmail.com>]

Thanks, I have one follow-up:

On Tue, Jun 4, 2013 at 12:10 AM, Nitin Mehta <Ni...@citrix.com> wrote:
> All the public templates will be replicated to all the secondary storage
> for redundancy but private templates and uploaded volumes should be kept
> in one of the randomly chosen secondary storage. Snapshots will be
> randomly copied to one of the secondary storage (the chain of incremental
> snapshots are kept on the same secondary storage.). Currently these
> algorithms are hardcoded and so in case you need flexibility please open
> an enhancement for the same.
>
>
> On 04/06/13 2:57 AM, "Marcus Sorensen" <sh...@gmail.com> wrote:
>
>>Anyone?
>>
>>We are also noticing that if we register a template, that template is
>>registered and ready, but becomes unusable if the URL for the template
>>stops working at any time. Seems like we attempt to red download it every
>>so often. So no one-time registration. Is this by design?
>
> Whenever there is a handshake between MS and the agent running on the SSVM
> there is a template sync which syncs the template status on the MS DB and
> the template's physical Location and triggers the download if its not
> complete or retries in case there was some error. (This handshake happens
> on ssvm re/start, agent on SSVM re/start and MS re/start).

This implies that once the template is downloaded and is ready, the
URL doesn't need to stick around. We have some instances where we may
be downloading templates via one-time URLs. We had seen some things
that made us think that the template URLs are checked and possibly
redownloaded on occasion, so I wanted to clarify this, perhaps our
interpretations were wrong. Obviously if the URLs are checked and a
redownload is attempted for any reason, it will fail.

>
>
>>On May 31, 2013 2:35 PM, "Marcus Sorensen" <sh...@gmail.com> wrote:
>>
>>> Sort of embarrassed to ask this, but I'm unclear on a few of the
>>> aspects of secondary storage.
>>>
>>> 1) I can see that if I add multiple secondary storage locations, they
>>> each get a copy of the registered templates. Are these copied from one
>>> secondary storage to another, or from the url of the registered
>>> template?
>
> From the url of the template. Should happen only for public templates. If
> you see otherwise please file a bug.
>
>>>
>>> 2) It seems that the intent is to mirror templates across all
>>> secondary storages, zone-wide, correct? But no copies are made for the
>>> snapshots or volumes dirs. This may point to the idea that templates
>>> are always downloaded via URLs
>
> Multiple sec. storage are added for adding more storage and making public
> templates highly available.
> Templates are downloaded via URLs but can also be created from snapshots
> and volumes
>
>>>
>>> 3) It sounds like the total size of secondary storage is somewhat
>>> limited by templates, e.g. if all of you secondary storage is in
>>> templates, adding another secondary storage server won't increase my
>>> capacity, correct?
>
> Not correct, as per explanation above.
>
>>>
>

Re: secondary storage questions

[Nitin Mehta <Ni...@citrix.com>]

All the public templates will be replicated to all the secondary storage
for redundancy but private templates and uploaded volumes should be kept
in one of the randomly chosen secondary storage. Snapshots will be
randomly copied to one of the secondary storage (the chain of incremental
snapshots are kept on the same secondary storage.). Currently these
algorithms are hardcoded and so in case you need flexibility please open
an enhancement for the same.


On 04/06/13 2:57 AM, "Marcus Sorensen" <sh...@gmail.com> wrote:

>Anyone?
>
>We are also noticing that if we register a template, that template is
>registered and ready, but becomes unusable if the URL for the template
>stops working at any time. Seems like we attempt to red download it every
>so often. So no one-time registration. Is this by design?

Whenever there is a handshake between MS and the agent running on the SSVM
there is a template sync which syncs the template status on the MS DB and
the template's physical Location and triggers the download if its not
complete or retries in case there was some error. (This handshake happens
on ssvm re/start, agent on SSVM re/start and MS re/start).


>On May 31, 2013 2:35 PM, "Marcus Sorensen" <sh...@gmail.com> wrote:
>
>> Sort of embarrassed to ask this, but I'm unclear on a few of the
>> aspects of secondary storage.
>>
>> 1) I can see that if I add multiple secondary storage locations, they
>> each get a copy of the registered templates. Are these copied from one
>> secondary storage to another, or from the url of the registered
>> template?

>From the url of the template. Should happen only for public templates. If
you see otherwise please file a bug.

>>
>> 2) It seems that the intent is to mirror templates across all
>> secondary storages, zone-wide, correct? But no copies are made for the
>> snapshots or volumes dirs. This may point to the idea that templates
>> are always downloaded via URLs

Multiple sec. storage are added for adding more storage and making public
templates highly available.
Templates are downloaded via URLs but can also be created from snapshots
and volumes

>>
>> 3) It sounds like the total size of secondary storage is somewhat
>> limited by templates, e.g. if all of you secondary storage is in
>> templates, adding another secondary storage server won't increase my
>> capacity, correct?

Not correct, as per explanation above.

>>

Re: secondary storage questions

[Marcus Sorensen <sh...@gmail.com>]

Anyone?

We are also noticing that if we register a template, that template is
registered and ready, but becomes unusable if the URL for the template
stops working at any time. Seems like we attempt to red download it every
so often. So no one-time registration. Is this by design?
On May 31, 2013 2:35 PM, "Marcus Sorensen" <sh...@gmail.com> wrote:

> Sort of embarrassed to ask this, but I'm unclear on a few of the
> aspects of secondary storage.
>
> 1) I can see that if I add multiple secondary storage locations, they
> each get a copy of the registered templates. Are these copied from one
> secondary storage to another, or from the url of the registered
> template?
>
> 2) It seems that the intent is to mirror templates across all
> secondary storages, zone-wide, correct? But no copies are made for the
> snapshots or volumes dirs. This may point to the idea that templates
> are always downloaded via URLs
>
> 3) It sounds like the total size of secondary storage is somewhat
> limited by templates, e.g. if all of you secondary storage is in
> templates, adding another secondary storage server won't increase my
> capacity, correct?
>