You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@cloudstack.apache.org by Richard Downer <ri...@apache.org> on 2017/09/29 12:21:45 UTC
listTemplates: multiple items with same ID
Hello,
I'm debugging an issue on behalf of one of our users. We are using Apache jclouds to talk to Apache CloudStack which is on a customer's site.
jclouds is invoking the `listTemplates` API, but is then choking while processing the query results. It seems that on this CloudStack instance, there are multiple templates with same ID - these all seem to refer to the same OS image, but in different zones - and jclouds doesn't like it when things have the same ID but different content.
Digging through the email archives, I found this message:
http://mail-archives.apache.org/mod_mbox/cloudstack-dev/201501.mbox/%3C54C6C184.3010306@gmail.com%3E
..which suggests that if a template is cross-zone, then it has the same ID in all zones.
However the templates in this case all have "crossZones":false, and the above linked message sets the expectation that templates will NOT have the same ID.
Could somebody confirm the expectations for uniqueness of template IDs? Is it guaranteed that (under normal circumstances) template IDs are unique except for when crossZones is set - or is duplicate template IDs always a possibility? Does jclouds need to change its understanding of the uniqueness of template IDs, or is there something odd about this customer's CloudStack installation?
Thanks!
Richard.
Re: listTemplates: multiple items with same ID
Posted by Sergey Levitskiy <se...@hotmail.com>.
As long as template is copied from zone to zone it will present with duplicate ID. This is by design. crossZone parameter is to indicate to auto replicate template across all available zone on template creation. To put it differently only combination of template ID and zone ID is a primary key for template querying.
Thanks,
Sergey
Re: listTemplates: multiple items with same ID
Posted by Rafael Weingärtner <ra...@gmail.com>.
How many zones does this ACS deployment have? Also, would it be possible
for you to post some these duplicated entries for us here?
On Mon, Oct 2, 2017 at 8:40 AM, Richard Downer <ri...@apache.org> wrote:
> Rafael,
>
> I have a sample `listTemplates` output. It's a real mixed bag. 5 templates
> with crossZones==false have duplicated IDs, but duplicated a different
> number of times. There are 4 instances of "GNU/Linux Fedora 20 - Minimal -
> 64bits", 14 instances of "GNU/Linux Ubuntu 14.04 - Minimal - 64bits"; other
> images duplicated 5 times or 8 times.
>
> If it is relevant - in each case, there are no other instances of the
> template with other IDs (i.e. there are 4 "GNU/Linux Fedora 20 - Minimal -
> 64bits" with a single ID, but no other templates with that name).
>
> Any theory on how this might have happened (database manual intervention)?
>
> If there's any other information you might find helpful please let me
> know, although I do not have direct access to the ACS instance in question
> (it's my client's client's ACS) so it may take me a little while to get
> answers.
>
> Thanks for your help!
>
> Richard.
>
>
> On 2017-10-02 11:31, Rafael Weingärtner <ra...@gmail.com>
> wrote:
> > I would consider this a valid assumption. I checked one of the ACS
> > environments (version 4.9.2, updated from 4.5) that I know use multiple
> > zones, and all of the templates that have IDs duplicated have
> > crossZones==true.
> >
> > We would need more information from this ACS that you are debugging to
> pin
> > point the problem. How many templates that have crossZones==false are
> > presenting this duplication problem? If it is a bug, I would expect
> > everyone of them to have duplicated IDs. However, if this is the only
> thing
> > they differ from other single zone templates, I would guess that these
> > templates suffered from manual intervention in the database.
> >
> > On Mon, Oct 2, 2017 at 7:22 AM, Richard Downer <ri...@apache.org>
> wrote:
> >
> > > Rafael,
> > >
> > > Thanks for the information. Is there any information in listTemplates
> > > which would indicate if a template is in multiple zones? e.g. is the
> > > `crossZones` parameter a reliable indicator - can we normally be
> guaranteed
> > > that there will be no duplicated IDs in templates where
> crossZones==false?
> > >
> > > Currently jclouds will expect all templates where crossZones==false to
> > > have unique IDs. I am trying to determine if this is a valid
> assumption, or
> > > if jclouds needs a bug fix to be prepared for a duplicate ID at any
> time.
> > >
> > > Many thanks
> > > Richard.
> > >
> > >
> > > On 2017-09-29 15:24, Rafael Weingärtner <ra...@gmail.com>
> > > wrote:
> > > > If the template is in multiple zones at the same time, then you are
> going
> > > > to have multiple template objects with the same ID. Otherwise, this
> > > should
> > > > not happen. You could check the database table "vm_template" to see
> if
> > > you
> > > > find any inconsistency there.
> > > >
> > > > On Fri, Sep 29, 2017 at 9:21 AM, Richard Downer <ri...@apache.org>
> > > wrote:
> > > >
> > > > > Hello,
> > > > >
> > > > > I'm debugging an issue on behalf of one of our users. We are using
> > > Apache
> > > > > jclouds to talk to Apache CloudStack which is on a customer's site.
> > > > >
> > > > > jclouds is invoking the `listTemplates` API, but is then choking
> while
> > > > > processing the query results. It seems that on this CloudStack
> > > instance,
> > > > > there are multiple templates with same ID - these all seem to
> refer to
> > > the
> > > > > same OS image, but in different zones - and jclouds doesn't like it
> > > when
> > > > > things have the same ID but different content.
> > > > >
> > > > > Digging through the email archives, I found this message:
> > > > > http://mail-archives.apache.org/mod_mbox/cloudstack-dev/
> > > > > 201501.mbox/%3C54C6C184.3010306@gmail.com%3E
> > > > > ..which suggests that if a template is cross-zone, then it has the
> > > same ID
> > > > > in all zones.
> > > > >
> > > > > However the templates in this case all have "crossZones":false,
> and the
> > > > > above linked message sets the expectation that templates will NOT
> have
> > > the
> > > > > same ID.
> > > > >
> > > > > Could somebody confirm the expectations for uniqueness of template
> > > IDs? Is
> > > > > it guaranteed that (under normal circumstances) template IDs are
> unique
> > > > > except for when crossZones is set - or is duplicate template IDs
> > > always a
> > > > > possibility? Does jclouds need to change its understanding of the
> > > > > uniqueness of template IDs, or is there something odd about this
> > > customer's
> > > > > CloudStack installation?
> > > > >
> > > > > Thanks!
> > > > >
> > > > > Richard.
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Rafael Weingärtner
> > > >
> > >
> >
> >
> >
> > --
> > Rafael Weingärtner
> >
>
--
Rafael Weingärtner
Re: listTemplates: multiple items with same ID
Posted by Richard Downer <ri...@apache.org>.
Rafael,
I have a sample `listTemplates` output. It's a real mixed bag. 5 templates with crossZones==false have duplicated IDs, but duplicated a different number of times. There are 4 instances of "GNU/Linux Fedora 20 - Minimal - 64bits", 14 instances of "GNU/Linux Ubuntu 14.04 - Minimal - 64bits"; other images duplicated 5 times or 8 times.
If it is relevant - in each case, there are no other instances of the template with other IDs (i.e. there are 4 "GNU/Linux Fedora 20 - Minimal - 64bits" with a single ID, but no other templates with that name).
Any theory on how this might have happened (database manual intervention)?
If there's any other information you might find helpful please let me know, although I do not have direct access to the ACS instance in question (it's my client's client's ACS) so it may take me a little while to get answers.
Thanks for your help!
Richard.
On 2017-10-02 11:31, Rafael Weingärtner <ra...@gmail.com> wrote:
> I would consider this a valid assumption. I checked one of the ACS
> environments (version 4.9.2, updated from 4.5) that I know use multiple
> zones, and all of the templates that have IDs duplicated have
> crossZones==true.
>
> We would need more information from this ACS that you are debugging to pin
> point the problem. How many templates that have crossZones==false are
> presenting this duplication problem? If it is a bug, I would expect
> everyone of them to have duplicated IDs. However, if this is the only thing
> they differ from other single zone templates, I would guess that these
> templates suffered from manual intervention in the database.
>
> On Mon, Oct 2, 2017 at 7:22 AM, Richard Downer <ri...@apache.org> wrote:
>
> > Rafael,
> >
> > Thanks for the information. Is there any information in listTemplates
> > which would indicate if a template is in multiple zones? e.g. is the
> > `crossZones` parameter a reliable indicator - can we normally be guaranteed
> > that there will be no duplicated IDs in templates where crossZones==false?
> >
> > Currently jclouds will expect all templates where crossZones==false to
> > have unique IDs. I am trying to determine if this is a valid assumption, or
> > if jclouds needs a bug fix to be prepared for a duplicate ID at any time.
> >
> > Many thanks
> > Richard.
> >
> >
> > On 2017-09-29 15:24, Rafael Weingärtner <ra...@gmail.com>
> > wrote:
> > > If the template is in multiple zones at the same time, then you are going
> > > to have multiple template objects with the same ID. Otherwise, this
> > should
> > > not happen. You could check the database table "vm_template" to see if
> > you
> > > find any inconsistency there.
> > >
> > > On Fri, Sep 29, 2017 at 9:21 AM, Richard Downer <ri...@apache.org>
> > wrote:
> > >
> > > > Hello,
> > > >
> > > > I'm debugging an issue on behalf of one of our users. We are using
> > Apache
> > > > jclouds to talk to Apache CloudStack which is on a customer's site.
> > > >
> > > > jclouds is invoking the `listTemplates` API, but is then choking while
> > > > processing the query results. It seems that on this CloudStack
> > instance,
> > > > there are multiple templates with same ID - these all seem to refer to
> > the
> > > > same OS image, but in different zones - and jclouds doesn't like it
> > when
> > > > things have the same ID but different content.
> > > >
> > > > Digging through the email archives, I found this message:
> > > > http://mail-archives.apache.org/mod_mbox/cloudstack-dev/
> > > > 201501.mbox/%3C54C6C184.3010306@gmail.com%3E
> > > > ..which suggests that if a template is cross-zone, then it has the
> > same ID
> > > > in all zones.
> > > >
> > > > However the templates in this case all have "crossZones":false, and the
> > > > above linked message sets the expectation that templates will NOT have
> > the
> > > > same ID.
> > > >
> > > > Could somebody confirm the expectations for uniqueness of template
> > IDs? Is
> > > > it guaranteed that (under normal circumstances) template IDs are unique
> > > > except for when crossZones is set - or is duplicate template IDs
> > always a
> > > > possibility? Does jclouds need to change its understanding of the
> > > > uniqueness of template IDs, or is there something odd about this
> > customer's
> > > > CloudStack installation?
> > > >
> > > > Thanks!
> > > >
> > > > Richard.
> > > >
> > >
> > >
> > >
> > > --
> > > Rafael Weingärtner
> > >
> >
>
>
>
> --
> Rafael Weingärtner
>
Re: listTemplates: multiple items with same ID
Posted by Rafael Weingärtner <ra...@gmail.com>.
I would consider this a valid assumption. I checked one of the ACS
environments (version 4.9.2, updated from 4.5) that I know use multiple
zones, and all of the templates that have IDs duplicated have
crossZones==true.
We would need more information from this ACS that you are debugging to pin
point the problem. How many templates that have crossZones==false are
presenting this duplication problem? If it is a bug, I would expect
everyone of them to have duplicated IDs. However, if this is the only thing
they differ from other single zone templates, I would guess that these
templates suffered from manual intervention in the database.
On Mon, Oct 2, 2017 at 7:22 AM, Richard Downer <ri...@apache.org> wrote:
> Rafael,
>
> Thanks for the information. Is there any information in listTemplates
> which would indicate if a template is in multiple zones? e.g. is the
> `crossZones` parameter a reliable indicator - can we normally be guaranteed
> that there will be no duplicated IDs in templates where crossZones==false?
>
> Currently jclouds will expect all templates where crossZones==false to
> have unique IDs. I am trying to determine if this is a valid assumption, or
> if jclouds needs a bug fix to be prepared for a duplicate ID at any time.
>
> Many thanks
> Richard.
>
>
> On 2017-09-29 15:24, Rafael Weingärtner <ra...@gmail.com>
> wrote:
> > If the template is in multiple zones at the same time, then you are going
> > to have multiple template objects with the same ID. Otherwise, this
> should
> > not happen. You could check the database table "vm_template" to see if
> you
> > find any inconsistency there.
> >
> > On Fri, Sep 29, 2017 at 9:21 AM, Richard Downer <ri...@apache.org>
> wrote:
> >
> > > Hello,
> > >
> > > I'm debugging an issue on behalf of one of our users. We are using
> Apache
> > > jclouds to talk to Apache CloudStack which is on a customer's site.
> > >
> > > jclouds is invoking the `listTemplates` API, but is then choking while
> > > processing the query results. It seems that on this CloudStack
> instance,
> > > there are multiple templates with same ID - these all seem to refer to
> the
> > > same OS image, but in different zones - and jclouds doesn't like it
> when
> > > things have the same ID but different content.
> > >
> > > Digging through the email archives, I found this message:
> > > http://mail-archives.apache.org/mod_mbox/cloudstack-dev/
> > > 201501.mbox/%3C54C6C184.3010306@gmail.com%3E
> > > ..which suggests that if a template is cross-zone, then it has the
> same ID
> > > in all zones.
> > >
> > > However the templates in this case all have "crossZones":false, and the
> > > above linked message sets the expectation that templates will NOT have
> the
> > > same ID.
> > >
> > > Could somebody confirm the expectations for uniqueness of template
> IDs? Is
> > > it guaranteed that (under normal circumstances) template IDs are unique
> > > except for when crossZones is set - or is duplicate template IDs
> always a
> > > possibility? Does jclouds need to change its understanding of the
> > > uniqueness of template IDs, or is there something odd about this
> customer's
> > > CloudStack installation?
> > >
> > > Thanks!
> > >
> > > Richard.
> > >
> >
> >
> >
> > --
> > Rafael Weingärtner
> >
>
--
Rafael Weingärtner
Re: listTemplates: multiple items with same ID
Posted by Richard Downer <ri...@apache.org>.
Rafael,
Thanks for the information. Is there any information in listTemplates which would indicate if a template is in multiple zones? e.g. is the `crossZones` parameter a reliable indicator - can we normally be guaranteed that there will be no duplicated IDs in templates where crossZones==false?
Currently jclouds will expect all templates where crossZones==false to have unique IDs. I am trying to determine if this is a valid assumption, or if jclouds needs a bug fix to be prepared for a duplicate ID at any time.
Many thanks
Richard.
On 2017-09-29 15:24, Rafael Weingärtner <ra...@gmail.com> wrote:
> If the template is in multiple zones at the same time, then you are going
> to have multiple template objects with the same ID. Otherwise, this should
> not happen. You could check the database table "vm_template" to see if you
> find any inconsistency there.
>
> On Fri, Sep 29, 2017 at 9:21 AM, Richard Downer <ri...@apache.org> wrote:
>
> > Hello,
> >
> > I'm debugging an issue on behalf of one of our users. We are using Apache
> > jclouds to talk to Apache CloudStack which is on a customer's site.
> >
> > jclouds is invoking the `listTemplates` API, but is then choking while
> > processing the query results. It seems that on this CloudStack instance,
> > there are multiple templates with same ID - these all seem to refer to the
> > same OS image, but in different zones - and jclouds doesn't like it when
> > things have the same ID but different content.
> >
> > Digging through the email archives, I found this message:
> > http://mail-archives.apache.org/mod_mbox/cloudstack-dev/
> > 201501.mbox/%3C54C6C184.3010306@gmail.com%3E
> > ..which suggests that if a template is cross-zone, then it has the same ID
> > in all zones.
> >
> > However the templates in this case all have "crossZones":false, and the
> > above linked message sets the expectation that templates will NOT have the
> > same ID.
> >
> > Could somebody confirm the expectations for uniqueness of template IDs? Is
> > it guaranteed that (under normal circumstances) template IDs are unique
> > except for when crossZones is set - or is duplicate template IDs always a
> > possibility? Does jclouds need to change its understanding of the
> > uniqueness of template IDs, or is there something odd about this customer's
> > CloudStack installation?
> >
> > Thanks!
> >
> > Richard.
> >
>
>
>
> --
> Rafael Weingärtner
>
Re: listTemplates: multiple items with same ID
Posted by Rafael Weingärtner <ra...@gmail.com>.
If the template is in multiple zones at the same time, then you are going
to have multiple template objects with the same ID. Otherwise, this should
not happen. You could check the database table "vm_template" to see if you
find any inconsistency there.
On Fri, Sep 29, 2017 at 9:21 AM, Richard Downer <ri...@apache.org> wrote:
> Hello,
>
> I'm debugging an issue on behalf of one of our users. We are using Apache
> jclouds to talk to Apache CloudStack which is on a customer's site.
>
> jclouds is invoking the `listTemplates` API, but is then choking while
> processing the query results. It seems that on this CloudStack instance,
> there are multiple templates with same ID - these all seem to refer to the
> same OS image, but in different zones - and jclouds doesn't like it when
> things have the same ID but different content.
>
> Digging through the email archives, I found this message:
> http://mail-archives.apache.org/mod_mbox/cloudstack-dev/
> 201501.mbox/%3C54C6C184.3010306@gmail.com%3E
> ..which suggests that if a template is cross-zone, then it has the same ID
> in all zones.
>
> However the templates in this case all have "crossZones":false, and the
> above linked message sets the expectation that templates will NOT have the
> same ID.
>
> Could somebody confirm the expectations for uniqueness of template IDs? Is
> it guaranteed that (under normal circumstances) template IDs are unique
> except for when crossZones is set - or is duplicate template IDs always a
> possibility? Does jclouds need to change its understanding of the
> uniqueness of template IDs, or is there something odd about this customer's
> CloudStack installation?
>
> Thanks!
>
> Richard.
>
--
Rafael Weingärtner