You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@jclouds.apache.org by Mikołaj Zalewski <mi...@google.com> on 2014/04/25 13:26:28 UTC
Best way to handle GCE's obsolete machine types?
Hi,
When working with jclouds I've stumbled on a problem during GCE VM
creation. If I specify machine hardware by constraints, the framework can
find an obsolete hardware profile and the creation will fail (an obsolete
hardware profile in GCE means that one can't create new instance of this
platform, but there may still be instances running). I've opened JIRA
550<https://issues.apache.org/jira/browse/JCLOUDS-550>for it. What's
the recommended way to fix this? I can think of three ways:
(a) don't advertise obsolete (and deleted) machine types in
computeService.listHardwareProfiles()
at all.
(b) add the notion of an obsolete (as well as deleted and deprecated?)
profile to the base Hardware object and use it in TemplateBuilderImpl to
filter out these profiles.
(c) try to use some subclassing/injections for the TemplateBuilder to
work differently for GCE than for others and to know about the hardware
states.
I personally don't like (c) while as for (a) and (b) I don't have the
experience about possible side-effects to choose one. What's you advise
which solution is the best?
Mikołaj Zalewski
PS: choosing one of the hardware profiles from
computeService.listHardwareProfiles()
and passing it to TemplateBuilder.fromHardware() doesn't necessarily lead
to this profile being chosen, as only some fields from the parameter are
used as constraints and the id is not one of them. Is this a bug or a
feature?
Re: Best way to handle GCE's obsolete machine types?
Posted by Ignasi Barrera <na...@apache.org>.
Agree that has some disadvantages. As far as I understand the compute service:
* The method that lists the hardware profiles should only return the
ones that can be used.
* Nodes should provide enough information that allows to operate them
(show their capacity, grow them if supported, scale, etc).
The obsolete hardwares, however, seem to be incompatible with this. If
we return them when listing hardware profiles, the template builder
may select hardwares that won't work, and if we don't return them, we
could have nodes which hardware is not present in the hardware profile
list (and I don't know if that could have side effects, although I
can't think about any right now). A user could try to reuse that
hardware using the "hardwareId" method in the template builder, and
that will fail even when the hardware information was provided by the
compute service itself (not directly with the listHardwareProfiles
method, but with the getNodeMetadata, for example).
With this in mind, let's be creative and think about how we'd like the
compute service to behave! :)
One possible approach would be to not return the obsolete hardware
profiles when listing them, and populate the hardware info *without*
the ID when populating obsolete hardwares in the node. That would
prevent using the "hardwareId" method in the template builder, but the
"fromHardware" approach should be able to find an equivalent one to
deploy. The counterpart of this approach is that hardwares are assumed
to have an ID, and leaving it to null can potentially generate NPEs.
This are just some ideas that come to my mind after thinking a bit
about this, and I would love to see other proposals and ideas! Let's
design the solution right :)
Regarding the pull request thing, shame on me! I completely forgot
about it after adding the initial comment. Will review it this
weekend.
We are a young community and still don't have strong/formal
procedures. Feedback and suggestions from the community are very
welcome! Don't hesitate to ask questions, ping us to get the
attention, and, in general, ask for everything about the project:
code, procedures, whatever!
HTH!
Ignasi
On 25 April 2014 18:11, Mikołaj Zalewski <mi...@google.com> wrote:
> On Fri, Apr 25, 2014 at 3:36 PM, Ignasi Barrera <na...@apache.org> wrote:
>
>> Hi,
>>
>> If the obsolete hardware profiles can't be used, I think the
>> "listHardwareProfiles" should not return them (which is approach (a)).
>>
>> This, however, has some implications: how should the "listNodes"
>> behave if there are existing nodes that were created using that
>> hardware profiles?
>>
>> The NodeMetadata [1] class defines its "hardware" property as
>> "nullable", so if the "listHardwareProfiles" method does not return
>> the obsolete hardware profiles, one can assume that the nodes created
>> using obsolete hardawre profiles won't have that property set. There
>> has been a recent discussion on a similar topic in the DigitalOcean
>> provider [2], and this is the way it is being addressed. Furthermore,
>> that node can't be deployed again with the exact same configuration,
>> as the hardware profile is obsolete, so I think this approach would
>> make sense. Would love to hear more opinions on this, though! :)
>>
> Does this means that if I have obsolete platforms running, I can't write
> a program to tell me how much CPUs in total am I using? That would be a
> small disadvantage, but on the other hand the simplicity of this solution
> is an advantage.
>
>
>> Regarding your question about the "fromHardware" method (here are the
>> definition [3] and the default implementation [4], for reference), my
>> understanding is that the "fromHardware" method's purpose is to define
>> a template that has an "equivalent" hardware configuration, but
>> there's no need for it to be exactly the same one. If one wants a
>> concrete hardware profile, the "hardwareId" method in the
>> TemplateBuilder should be used instead. Does this make sense to you?
>>
> This was very surprising when I first observed it, but it does have it's
> logic (e.g. it allows to use Hardware from one provider to create a similar
> instance with another).
>
>
>>
>>
>> Ignasi
>>
> Mikołaj
> PS: I don't know the process of getting patches merged yet - in my pull
> request [1] there is no more activity since a few days. Should I just wait
> or is it my turn to do something?
>
> [1] https://github.com/jclouds/jclouds-examples/pull/44
>
>
>>
>>
>> [1]
>> https://github.com/jclouds/jclouds/blob/master/compute/src/main/java/org/jclouds/compute/domain/NodeMetadata.java#L82-L87
>> [2] https://github.com/jclouds/jclouds-labs/pull/58
>> [3]
>> https://github.com/jclouds/jclouds/blob/master/compute/src/main/java/org/jclouds/compute/domain/TemplateBuilder.java#L41-L44
>> [4]
>> https://github.com/jclouds/jclouds/blob/master/compute/src/main/java/org/jclouds/compute/domain/internal/TemplateBuilderImpl.java#L540-L552
>>
>> On 25 April 2014 15:27, Andrea Turli <an...@gmail.com> wrote:
>> > Hi,
>> >
>> > I've issued a PR to fix that issue:
>> > https://github.com/jclouds/jclouds-labs-google/pull/24
>> >
>> > Please review it and merge, if ok.
>> >
>> > Thanks,
>> > Andrea
>> >
>> > On Fri, Apr 25, 2014 at 1:49 PM, Andrea Turli <an...@gmail.com>
>> wrote:
>> >> Hi Mikolaj,
>> >>
>> >> thanks for reporting that. I think the best approach would be to make
>> >> jclouds aware of the obsolete machineTypes by considering the
>> >> deprecated field in the MachineType domain object, so (b) if I get you
>> >> right.
>> >>
>> >> I've already a patch that deals with it that I can submit asap.
>> >>
>> >> Best,
>> >> Andrea
>> >>
>> >>
>> >>
>> >> On Fri, Apr 25, 2014 at 1:26 PM, Mikołaj Zalewski <mi...@google.com>
>> wrote:
>> >>> Hi,
>> >>> When working with jclouds I've stumbled on a problem during GCE VM
>> >>> creation. If I specify machine hardware by constraints, the framework
>> can
>> >>> find an obsolete hardware profile and the creation will fail (an
>> obsolete
>> >>> hardware profile in GCE means that one can't create new instance of
>> this
>> >>> platform, but there may still be instances running). I've opened JIRA
>> >>> 550<https://issues.apache.org/jira/browse/JCLOUDS-550>for it. What's
>> >>> the recommended way to fix this? I can think of three ways:
>> >>> (a) don't advertise obsolete (and deleted) machine types in
>> >>> computeService.listHardwareProfiles()
>> >>> at all.
>> >>> (b) add the notion of an obsolete (as well as deleted and
>> deprecated?)
>> >>> profile to the base Hardware object and use it in TemplateBuilderImpl
>> to
>> >>> filter out these profiles.
>> >>> (c) try to use some subclassing/injections for the TemplateBuilder to
>> >>> work differently for GCE than for others and to know about the hardware
>> >>> states.
>> >>> I personally don't like (c) while as for (a) and (b) I don't have the
>> >>> experience about possible side-effects to choose one. What's you advise
>> >>> which solution is the best?
>> >>>
>> >>> Mikołaj Zalewski
>> >>>
>> >>> PS: choosing one of the hardware profiles from
>> >>> computeService.listHardwareProfiles()
>> >>> and passing it to TemplateBuilder.fromHardware() doesn't necessarily
>> lead
>> >>> to this profile being chosen, as only some fields from the parameter
>> are
>> >>> used as constraints and the id is not one of them. Is this a bug or a
>> >>> feature?
>>
Re: Best way to handle GCE's obsolete machine types?
Posted by Mikołaj Zalewski <mi...@google.com>.
On Fri, Apr 25, 2014 at 3:36 PM, Ignasi Barrera <na...@apache.org> wrote:
> Hi,
>
> If the obsolete hardware profiles can't be used, I think the
> "listHardwareProfiles" should not return them (which is approach (a)).
>
> This, however, has some implications: how should the "listNodes"
> behave if there are existing nodes that were created using that
> hardware profiles?
>
> The NodeMetadata [1] class defines its "hardware" property as
> "nullable", so if the "listHardwareProfiles" method does not return
> the obsolete hardware profiles, one can assume that the nodes created
> using obsolete hardawre profiles won't have that property set. There
> has been a recent discussion on a similar topic in the DigitalOcean
> provider [2], and this is the way it is being addressed. Furthermore,
> that node can't be deployed again with the exact same configuration,
> as the hardware profile is obsolete, so I think this approach would
> make sense. Would love to hear more opinions on this, though! :)
>
Does this means that if I have obsolete platforms running, I can't write
a program to tell me how much CPUs in total am I using? That would be a
small disadvantage, but on the other hand the simplicity of this solution
is an advantage.
> Regarding your question about the "fromHardware" method (here are the
> definition [3] and the default implementation [4], for reference), my
> understanding is that the "fromHardware" method's purpose is to define
> a template that has an "equivalent" hardware configuration, but
> there's no need for it to be exactly the same one. If one wants a
> concrete hardware profile, the "hardwareId" method in the
> TemplateBuilder should be used instead. Does this make sense to you?
>
This was very surprising when I first observed it, but it does have it's
logic (e.g. it allows to use Hardware from one provider to create a similar
instance with another).
>
>
> Ignasi
>
Mikołaj
PS: I don't know the process of getting patches merged yet - in my pull
request [1] there is no more activity since a few days. Should I just wait
or is it my turn to do something?
[1] https://github.com/jclouds/jclouds-examples/pull/44
>
>
> [1]
> https://github.com/jclouds/jclouds/blob/master/compute/src/main/java/org/jclouds/compute/domain/NodeMetadata.java#L82-L87
> [2] https://github.com/jclouds/jclouds-labs/pull/58
> [3]
> https://github.com/jclouds/jclouds/blob/master/compute/src/main/java/org/jclouds/compute/domain/TemplateBuilder.java#L41-L44
> [4]
> https://github.com/jclouds/jclouds/blob/master/compute/src/main/java/org/jclouds/compute/domain/internal/TemplateBuilderImpl.java#L540-L552
>
> On 25 April 2014 15:27, Andrea Turli <an...@gmail.com> wrote:
> > Hi,
> >
> > I've issued a PR to fix that issue:
> > https://github.com/jclouds/jclouds-labs-google/pull/24
> >
> > Please review it and merge, if ok.
> >
> > Thanks,
> > Andrea
> >
> > On Fri, Apr 25, 2014 at 1:49 PM, Andrea Turli <an...@gmail.com>
> wrote:
> >> Hi Mikolaj,
> >>
> >> thanks for reporting that. I think the best approach would be to make
> >> jclouds aware of the obsolete machineTypes by considering the
> >> deprecated field in the MachineType domain object, so (b) if I get you
> >> right.
> >>
> >> I've already a patch that deals with it that I can submit asap.
> >>
> >> Best,
> >> Andrea
> >>
> >>
> >>
> >> On Fri, Apr 25, 2014 at 1:26 PM, Mikołaj Zalewski <mi...@google.com>
> wrote:
> >>> Hi,
> >>> When working with jclouds I've stumbled on a problem during GCE VM
> >>> creation. If I specify machine hardware by constraints, the framework
> can
> >>> find an obsolete hardware profile and the creation will fail (an
> obsolete
> >>> hardware profile in GCE means that one can't create new instance of
> this
> >>> platform, but there may still be instances running). I've opened JIRA
> >>> 550<https://issues.apache.org/jira/browse/JCLOUDS-550>for it. What's
> >>> the recommended way to fix this? I can think of three ways:
> >>> (a) don't advertise obsolete (and deleted) machine types in
> >>> computeService.listHardwareProfiles()
> >>> at all.
> >>> (b) add the notion of an obsolete (as well as deleted and
> deprecated?)
> >>> profile to the base Hardware object and use it in TemplateBuilderImpl
> to
> >>> filter out these profiles.
> >>> (c) try to use some subclassing/injections for the TemplateBuilder to
> >>> work differently for GCE than for others and to know about the hardware
> >>> states.
> >>> I personally don't like (c) while as for (a) and (b) I don't have the
> >>> experience about possible side-effects to choose one. What's you advise
> >>> which solution is the best?
> >>>
> >>> Mikołaj Zalewski
> >>>
> >>> PS: choosing one of the hardware profiles from
> >>> computeService.listHardwareProfiles()
> >>> and passing it to TemplateBuilder.fromHardware() doesn't necessarily
> lead
> >>> to this profile being chosen, as only some fields from the parameter
> are
> >>> used as constraints and the id is not one of them. Is this a bug or a
> >>> feature?
>
Re: Best way to handle GCE's obsolete machine types?
Posted by Ignasi Barrera <na...@apache.org>.
Hi,
If the obsolete hardware profiles can't be used, I think the
"listHardwareProfiles" should not return them (which is approach (a)).
This, however, has some implications: how should the "listNodes"
behave if there are existing nodes that were created using that
hardware profiles?
The NodeMetadata [1] class defines its "hardware" property as
"nullable", so if the "listHardwareProfiles" method does not return
the obsolete hardware profiles, one can assume that the nodes created
using obsolete hardawre profiles won't have that property set. There
has been a recent discussion on a similar topic in the DigitalOcean
provider [2], and this is the way it is being addressed. Furthermore,
that node can't be deployed again with the exact same configuration,
as the hardware profile is obsolete, so I think this approach would
make sense. Would love to hear more opinions on this, though! :)
Regarding your question about the "fromHardware" method (here are the
definition [3] and the default implementation [4], for reference), my
understanding is that the "fromHardware" method's purpose is to define
a template that has an "equivalent" hardware configuration, but
there's no need for it to be exactly the same one. If one wants a
concrete hardware profile, the "hardwareId" method in the
TemplateBuilder should be used instead. Does this make sense to you?
Ignasi
[1] https://github.com/jclouds/jclouds/blob/master/compute/src/main/java/org/jclouds/compute/domain/NodeMetadata.java#L82-L87
[2] https://github.com/jclouds/jclouds-labs/pull/58
[3] https://github.com/jclouds/jclouds/blob/master/compute/src/main/java/org/jclouds/compute/domain/TemplateBuilder.java#L41-L44
[4] https://github.com/jclouds/jclouds/blob/master/compute/src/main/java/org/jclouds/compute/domain/internal/TemplateBuilderImpl.java#L540-L552
On 25 April 2014 15:27, Andrea Turli <an...@gmail.com> wrote:
> Hi,
>
> I've issued a PR to fix that issue:
> https://github.com/jclouds/jclouds-labs-google/pull/24
>
> Please review it and merge, if ok.
>
> Thanks,
> Andrea
>
> On Fri, Apr 25, 2014 at 1:49 PM, Andrea Turli <an...@gmail.com> wrote:
>> Hi Mikolaj,
>>
>> thanks for reporting that. I think the best approach would be to make
>> jclouds aware of the obsolete machineTypes by considering the
>> deprecated field in the MachineType domain object, so (b) if I get you
>> right.
>>
>> I've already a patch that deals with it that I can submit asap.
>>
>> Best,
>> Andrea
>>
>>
>>
>> On Fri, Apr 25, 2014 at 1:26 PM, Mikołaj Zalewski <mi...@google.com> wrote:
>>> Hi,
>>> When working with jclouds I've stumbled on a problem during GCE VM
>>> creation. If I specify machine hardware by constraints, the framework can
>>> find an obsolete hardware profile and the creation will fail (an obsolete
>>> hardware profile in GCE means that one can't create new instance of this
>>> platform, but there may still be instances running). I've opened JIRA
>>> 550<https://issues.apache.org/jira/browse/JCLOUDS-550>for it. What's
>>> the recommended way to fix this? I can think of three ways:
>>> (a) don't advertise obsolete (and deleted) machine types in
>>> computeService.listHardwareProfiles()
>>> at all.
>>> (b) add the notion of an obsolete (as well as deleted and deprecated?)
>>> profile to the base Hardware object and use it in TemplateBuilderImpl to
>>> filter out these profiles.
>>> (c) try to use some subclassing/injections for the TemplateBuilder to
>>> work differently for GCE than for others and to know about the hardware
>>> states.
>>> I personally don't like (c) while as for (a) and (b) I don't have the
>>> experience about possible side-effects to choose one. What's you advise
>>> which solution is the best?
>>>
>>> Mikołaj Zalewski
>>>
>>> PS: choosing one of the hardware profiles from
>>> computeService.listHardwareProfiles()
>>> and passing it to TemplateBuilder.fromHardware() doesn't necessarily lead
>>> to this profile being chosen, as only some fields from the parameter are
>>> used as constraints and the id is not one of them. Is this a bug or a
>>> feature?
Re: Best way to handle GCE's obsolete machine types?
Posted by Andrea Turli <an...@gmail.com>.
Hi,
I've issued a PR to fix that issue:
https://github.com/jclouds/jclouds-labs-google/pull/24
Please review it and merge, if ok.
Thanks,
Andrea
On Fri, Apr 25, 2014 at 1:49 PM, Andrea Turli <an...@gmail.com> wrote:
> Hi Mikolaj,
>
> thanks for reporting that. I think the best approach would be to make
> jclouds aware of the obsolete machineTypes by considering the
> deprecated field in the MachineType domain object, so (b) if I get you
> right.
>
> I've already a patch that deals with it that I can submit asap.
>
> Best,
> Andrea
>
>
>
> On Fri, Apr 25, 2014 at 1:26 PM, Mikołaj Zalewski <mi...@google.com> wrote:
>> Hi,
>> When working with jclouds I've stumbled on a problem during GCE VM
>> creation. If I specify machine hardware by constraints, the framework can
>> find an obsolete hardware profile and the creation will fail (an obsolete
>> hardware profile in GCE means that one can't create new instance of this
>> platform, but there may still be instances running). I've opened JIRA
>> 550<https://issues.apache.org/jira/browse/JCLOUDS-550>for it. What's
>> the recommended way to fix this? I can think of three ways:
>> (a) don't advertise obsolete (and deleted) machine types in
>> computeService.listHardwareProfiles()
>> at all.
>> (b) add the notion of an obsolete (as well as deleted and deprecated?)
>> profile to the base Hardware object and use it in TemplateBuilderImpl to
>> filter out these profiles.
>> (c) try to use some subclassing/injections for the TemplateBuilder to
>> work differently for GCE than for others and to know about the hardware
>> states.
>> I personally don't like (c) while as for (a) and (b) I don't have the
>> experience about possible side-effects to choose one. What's you advise
>> which solution is the best?
>>
>> Mikołaj Zalewski
>>
>> PS: choosing one of the hardware profiles from
>> computeService.listHardwareProfiles()
>> and passing it to TemplateBuilder.fromHardware() doesn't necessarily lead
>> to this profile being chosen, as only some fields from the parameter are
>> used as constraints and the id is not one of them. Is this a bug or a
>> feature?
Re: Best way to handle GCE's obsolete machine types?
Posted by Andrea Turli <an...@gmail.com>.
Hi Mikolaj,
thanks for reporting that. I think the best approach would be to make
jclouds aware of the obsolete machineTypes by considering the
deprecated field in the MachineType domain object, so (b) if I get you
right.
I've already a patch that deals with it that I can submit asap.
Best,
Andrea
On Fri, Apr 25, 2014 at 1:26 PM, Mikołaj Zalewski <mi...@google.com> wrote:
> Hi,
> When working with jclouds I've stumbled on a problem during GCE VM
> creation. If I specify machine hardware by constraints, the framework can
> find an obsolete hardware profile and the creation will fail (an obsolete
> hardware profile in GCE means that one can't create new instance of this
> platform, but there may still be instances running). I've opened JIRA
> 550<https://issues.apache.org/jira/browse/JCLOUDS-550>for it. What's
> the recommended way to fix this? I can think of three ways:
> (a) don't advertise obsolete (and deleted) machine types in
> computeService.listHardwareProfiles()
> at all.
> (b) add the notion of an obsolete (as well as deleted and deprecated?)
> profile to the base Hardware object and use it in TemplateBuilderImpl to
> filter out these profiles.
> (c) try to use some subclassing/injections for the TemplateBuilder to
> work differently for GCE than for others and to know about the hardware
> states.
> I personally don't like (c) while as for (a) and (b) I don't have the
> experience about possible side-effects to choose one. What's you advise
> which solution is the best?
>
> Mikołaj Zalewski
>
> PS: choosing one of the hardware profiles from
> computeService.listHardwareProfiles()
> and passing it to TemplateBuilder.fromHardware() doesn't necessarily lead
> to this profile being chosen, as only some fields from the parameter are
> used as constraints and the id is not one of them. Is this a bug or a
> feature?