You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@jclouds.apache.org by Iván Lomba <iv...@gmail.com> on 2016/06/13 11:45:40 UTC
Re: JCLOUDS-482 - Add support for arbitrary CPU and RAM in the
ComputeService
Hi!
I'm going to start with the implementation to support cpu and ram.
According with discussed here before, I understand that the best
approach will be to have auto-generated hardwares, and to take advantage
of minCores and minRAM to avoid interface changes and casts.
Regarding the ids, I'm not sure about what is better or what has more
sense: if using randomIds, parseableIds or even combine it? What do you
think?
Parseable ids that allow to reconstruct the hardware from the id seems
to me an useful option. If we decide to use that approach, what format
can we use? Something like:
"automatic-cpu-5-ram-2048..."
Regards.
Re: JCLOUDS-482 - Add support for arbitrary CPU and RAM in the ComputeService
Posted by Ignasi Barrera <na...@apache.org>.
> preconfigured... So providers like google which support both would,
> inherently no longer use it's preconfigured hardware profiles.
But many providers specify the disk size and others in the hardware
profile. It is not the case of GCE, but since we're implementing this
in compute-core, the solution must fit well in all providers, even
when the "explicit hardware profile" just includes CPU and RAM right
now (to keep the scope of the GSoC project bounded).
Re: JCLOUDS-482 - Add support for arbitrary CPU and RAM in the ComputeService
Posted by Iván Lomba <iv...@gmail.com>.
Hi!
After talking with Ignasi about what hardware should be resolved when a
provider support fixed values and hardware profiles (like Google), the best
option seems to be that unless user specify the hardwareId(with the
existent hardware) the template builder always create a fixed hardware.
Other options will be introduce some flag to determine whether or not use
hardware profiles, or to configure an "acceptable" distance to the nearest
hardware profile, and only use hardware profiles inside of that range, but
both options can be confusing for users.
Regards!
2016-06-28 7:41 GMT+02:00 Ignasi Barrera <na...@apache.org>:
> > +1 to this. Ideally, it would be easy for the user to discover what that
> profile is ("listHardwares" shouldn't really return it since it's not
> actually a "real" provider profile?), and perhaps even to configure it
> themselves. The latter is something we can leave as a provider-specific
> implementation ("jclouds.my-provider.default-profile-json" or whatever);
> not sure about the issue of how a user would discover what that profile is,
> though?
>
> I was thinking about binding it to the Guice context. Users will be able to
> override it and the binding would be just set by the providers supporting
> this. We could create a Guice module with the defaults and let the
> providers include it or not.
>
> > Would the following work here:
> >
> > 1) Use the existing resolution logic to pick the "best match", just as if
> the provider didn't support arbitrary attributes
> > 2) For those attributes where the provider *does* support arbitrary
> values, override the values in the profile from step 1) to match the user's
> request
> >
> > ? This should result in the "closest profile possible" to the user's
> request that the provider can support, which I think is what we are
> interpreting the template to mean?
>
> The only inconvenient I see in this approach is that we'll have to
> potentially change the id to the auto-generated one, and perhaps providers
> do something more than just configuring the values we pass in the profile
> object. This would effectively mean that we won't be using the
> preconfigured profiles at all, and we might be missing hardware
> optimizations. Also custom configurations might have different pricings. In
> any case, I see this as a good starting point.
>
> As an example, in Google Compute Engine you just pass the URL of the
> profile, being it a specially encoded one when using custom values (see the
> machineType parameter at [1]), so there is actually only a profile id being
> passed.
>
> P.S. @ivan we'll have to see how instances with custom machineTypes are
> returned and how jclouds shows them now, to understant what we have to
> adjust there.
>
> Also, once we have the basics in place, the Google implementation would
> have to do something to refine the user criteria to build the closest valid
> profile that meets all the restrictions [2] (but let's get the basic
> behavior working first).
>
> [1]
> https://cloud.google.com/compute/docs/reference/beta/
> instances/insert#request-body
> [2]
> https://cloud.google.com/compute/docs/instances/
> creating-instance-with-custom-machine-type#create
>
Re: JCLOUDS-482 - Add support for arbitrary CPU and RAM in the ComputeService
Posted by Ignasi Barrera <na...@apache.org>.
> +1 to this. Ideally, it would be easy for the user to discover what that
profile is ("listHardwares" shouldn't really return it since it's not
actually a "real" provider profile?), and perhaps even to configure it
themselves. The latter is something we can leave as a provider-specific
implementation ("jclouds.my-provider.default-profile-json" or whatever);
not sure about the issue of how a user would discover what that profile is,
though?
I was thinking about binding it to the Guice context. Users will be able to
override it and the binding would be just set by the providers supporting
this. We could create a Guice module with the defaults and let the
providers include it or not.
> Would the following work here:
>
> 1) Use the existing resolution logic to pick the "best match", just as if
the provider didn't support arbitrary attributes
> 2) For those attributes where the provider *does* support arbitrary
values, override the values in the profile from step 1) to match the user's
request
>
> ? This should result in the "closest profile possible" to the user's
request that the provider can support, which I think is what we are
interpreting the template to mean?
The only inconvenient I see in this approach is that we'll have to
potentially change the id to the auto-generated one, and perhaps providers
do something more than just configuring the values we pass in the profile
object. This would effectively mean that we won't be using the
preconfigured profiles at all, and we might be missing hardware
optimizations. Also custom configurations might have different pricings. In
any case, I see this as a good starting point.
As an example, in Google Compute Engine you just pass the URL of the
profile, being it a specially encoded one when using custom values (see the
machineType parameter at [1]), so there is actually only a profile id being
passed.
P.S. @ivan we'll have to see how instances with custom machineTypes are
returned and how jclouds shows them now, to understant what we have to
adjust there.
Also, once we have the basics in place, the Google implementation would
have to do something to refine the user criteria to build the closest valid
profile that meets all the restrictions [2] (but let's get the basic
behavior working first).
[1]
https://cloud.google.com/compute/docs/reference/beta/instances/insert#request-body
[2]
https://cloud.google.com/compute/docs/instances/creating-instance-with-custom-machine-type#create
Re: JCLOUDS-482 - Add support for arbitrary CPU and RAM in the
ComputeService
Posted by Andrew Phillips <an...@apache.org>.
> For the first question, I'd propose to have a default (just one)
> hardware profile for providers that don't support preconfigured
> profiles.
+1 to this. Ideally, it would be easy for the user to discover what that
profile is ("listHardwares" shouldn't really return it since it's not
actually a "real" provider profile?), and perhaps even to configure it
themselves. The latter is something we can leave as a provider-specific
implementation ("jclouds.my-provider.default-profile-json" or whatever);
not sure about the issue of how a user would discover what that profile
is, though?
> For the second question... I don't have a proposal yet :) Should we
> return the "closest" preconfigured hardware, or should we configure
> the explicit values set in the template builder?
Would the following work here:
1) Use the existing resolution logic to pick the "best match", just as
if the provider didn't support arbitrary attributes
2) For those attributes where the provider *does* support arbitrary
values, override the values in the profile from step 1) to match the
user's request
? This should result in the "closest profile possible" to the user's
request that the provider can support, which I think is what we are
interpreting the template to mean?
Regards
ap
Re: JCLOUDS-482 - Add support for arbitrary CPU and RAM in the ComputeService
Posted by Reijhanniel Jearl Campos <de...@gmail.com>.
Hello!
Been busy for quite a while, but here's to start again with my
contributions.. so here's my thoughts!
I think the *mantra* should be *less surprises*. So I agree with Ignasi's
proposal of returning just one hardware profile. This way, user's don't
have to guess (or be surprised) "how big" is "biggest()", "how fast" is
"fastest()", etc.
It is also ideal for providers to always prefer explicit values set (if
supported) in the template builder. I couldn't think of a use case where a
user might want to switch back and forth between using explicit and
preconfigured... So providers like google which support both would,
inherently no longer use it's preconfigured hardware profiles.
On Sun, Jun 26, 2016 at 6:19 AM Ignasi Barrera <na...@apache.org> wrote:
> I think it is OK to just implement a simple parser. The scope of the
> project is not to allow specifying complete hardware profiles, but
> adding basic support for arbitrary cpu and ram, which should set the
> foundation to implement the complete approach with little effort.
>
> Regarding the default value for CPU and RAM, I think it is OK to
> provide a "default hardware profile", but we have to think about the
> following use cases so we properly cover them in a way that we're all
> happy with.
>
> There are two premises:
>
> 1. Some providers have preconfigured hardware profiles but also
> support setting arbitrary CPU and RAM (Google is an example).
> 2. We assumed that providers that don't have hardware profiles would
> return an empty list in the "listHardwares" method.
>
> With these two premises in mind, we need to answer the following questions:
>
> 1. What hardware should be resolved when the user configures a
> template in one the following forms?
> - templateBuilder().build();
> - templateBuilder.smallest().build();
> - templateBuilder.biggest().build();
> - templateBuilder.fastest().build();
>
> 2. In a provider that has preconfigured hardware profiles and also
> supports explicitly setting arbitrary cpu/ram values, what hardware
> should be resolved when building a template like:
> - templateBuilder.minCores(2).minRam(2048).build();
>
> For the first question, I'd propose to have a default (just one)
> hardware profile for providers that don't support preconfigured
> profiles. All methods should then resolve the default one. For
> provider that support both approaches, like google, those methods
> should just consider the preconfigured profiles.
>
> For the second question... I don't have a proposal yet :) Should we
> return the "closest" preconfigured hardware, or should we configure
> the explicit values set in the template builder? Should we have a way
> to tell the template builder that we want a preconfigured profile or a
> custom one? (this would conflict with what we agreed to "reuse" the
> "minCores" and "minRam" methods).
>
>
> Please discuss! :)
>
>
>
> On 22 June 2016 at 20:01, Iván Lomba <lo...@gmail.com> wrote:
> > Hi!
> >
> > For the first approximation I've picked the format:
> > "automatic:cores=2;ram=1024"
> >
> > Initially in my first approach I assumed that the user specified both CPU
> > and RAM, but as Ignasi pointed at my first commit, maybe we should have
> > default values and let the user specify only
> > CPU
> > or RAM or even none. Thoughts about this?
> >
> > I did a small utility to parse automatic ids. In order to make it more
> > extensible I was thinking if use a
> > "parse
> > scheme
> > "
> > as in the hardwareSpec: different parsers with a Map that relates keys
> with
> > parsers, but it seems excessive to me for the moment, considering that
> only
> > ram and cores are parsed.
> >
> > BTW, I'm pushing at this branch:
> > https://github.com/ivanlomba/jclouds/tree/gsoc2016-JCLOUDS-482
> >
> > 2016-06-15 19:55 GMT+02:00 Ignasi Barrera <na...@apache.org>:
> >
> >> +1 to use parseable ids.
> >>
> >> Regarding the format, let's pick one that allows us to easily extend it
> in
> >> the future, in case we need to add there other hardware profile
> information
> >> such as disk sizes, etc, but keep it as simple as possible to cover the
> >> current use case.
> >>
> >> I.
> >> El 13 jun. 2016 1:47 p. m., "Iván Lomba" <iv...@gmail.com>
> escribió:
> >>
> >> > Hi!
> >> >
> >> > I'm going to start with the implementation to support cpu and ram.
> >> > According with discussed here before, I understand that the best
> approach
> >> > will be to have auto-generated hardwares, and to take advantage of
> >> minCores
> >> > and minRAM to avoid interface changes and casts.
> >> >
> >> > Regarding the ids, I'm not sure about what is better or what has more
> >> > sense: if using randomIds, parseableIds or even combine it? What do
> you
> >> > think?
> >> >
> >> > Parseable ids that allow to reconstruct the hardware from the id
> seems to
> >> > me an useful option. If we decide to use that approach, what format
> can
> >> we
> >> > use? Something like:
> >> >
> >> > "automatic-cpu-5-ram-2048..."
> >> >
> >> > Regards.
> >> >
> >>
>
Re: JCLOUDS-482 - Add support for arbitrary CPU and RAM in the ComputeService
Posted by Ignasi Barrera <na...@apache.org>.
I think it is OK to just implement a simple parser. The scope of the
project is not to allow specifying complete hardware profiles, but
adding basic support for arbitrary cpu and ram, which should set the
foundation to implement the complete approach with little effort.
Regarding the default value for CPU and RAM, I think it is OK to
provide a "default hardware profile", but we have to think about the
following use cases so we properly cover them in a way that we're all
happy with.
There are two premises:
1. Some providers have preconfigured hardware profiles but also
support setting arbitrary CPU and RAM (Google is an example).
2. We assumed that providers that don't have hardware profiles would
return an empty list in the "listHardwares" method.
With these two premises in mind, we need to answer the following questions:
1. What hardware should be resolved when the user configures a
template in one the following forms?
- templateBuilder().build();
- templateBuilder.smallest().build();
- templateBuilder.biggest().build();
- templateBuilder.fastest().build();
2. In a provider that has preconfigured hardware profiles and also
supports explicitly setting arbitrary cpu/ram values, what hardware
should be resolved when building a template like:
- templateBuilder.minCores(2).minRam(2048).build();
For the first question, I'd propose to have a default (just one)
hardware profile for providers that don't support preconfigured
profiles. All methods should then resolve the default one. For
provider that support both approaches, like google, those methods
should just consider the preconfigured profiles.
For the second question... I don't have a proposal yet :) Should we
return the "closest" preconfigured hardware, or should we configure
the explicit values set in the template builder? Should we have a way
to tell the template builder that we want a preconfigured profile or a
custom one? (this would conflict with what we agreed to "reuse" the
"minCores" and "minRam" methods).
Please discuss! :)
On 22 June 2016 at 20:01, Iván Lomba <lo...@gmail.com> wrote:
> Hi!
>
> For the first approximation I've picked the format:
> "automatic:cores=2;ram=1024"
>
> Initially in my first approach I assumed that the user specified both CPU
> and RAM, but as Ignasi pointed at my first commit, maybe we should have
> default values and let the user specify only
> CPU
> or RAM or even none. Thoughts about this?
>
> I did a small utility to parse automatic ids. In order to make it more
> extensible I was thinking if use a
> "parse
> scheme
> "
> as in the hardwareSpec: different parsers with a Map that relates keys with
> parsers, but it seems excessive to me for the moment, considering that only
> ram and cores are parsed.
>
> BTW, I'm pushing at this branch:
> https://github.com/ivanlomba/jclouds/tree/gsoc2016-JCLOUDS-482
>
> 2016-06-15 19:55 GMT+02:00 Ignasi Barrera <na...@apache.org>:
>
>> +1 to use parseable ids.
>>
>> Regarding the format, let's pick one that allows us to easily extend it in
>> the future, in case we need to add there other hardware profile information
>> such as disk sizes, etc, but keep it as simple as possible to cover the
>> current use case.
>>
>> I.
>> El 13 jun. 2016 1:47 p. m., "Iván Lomba" <iv...@gmail.com> escribió:
>>
>> > Hi!
>> >
>> > I'm going to start with the implementation to support cpu and ram.
>> > According with discussed here before, I understand that the best approach
>> > will be to have auto-generated hardwares, and to take advantage of
>> minCores
>> > and minRAM to avoid interface changes and casts.
>> >
>> > Regarding the ids, I'm not sure about what is better or what has more
>> > sense: if using randomIds, parseableIds or even combine it? What do you
>> > think?
>> >
>> > Parseable ids that allow to reconstruct the hardware from the id seems to
>> > me an useful option. If we decide to use that approach, what format can
>> we
>> > use? Something like:
>> >
>> > "automatic-cpu-5-ram-2048..."
>> >
>> > Regards.
>> >
>>
Re: JCLOUDS-482 - Add support for arbitrary CPU and RAM in the ComputeService
Posted by Iván Lomba <lo...@gmail.com>.
Hi!
For the first approximation I've picked the format:
"automatic:cores=2;ram=1024"
Initially in my first approach I assumed that the user specified both CPU
and RAM, but as Ignasi pointed at my first commit, maybe we should have
default values and let the user specify only
CPU
or RAM or even none. Thoughts about this?
I did a small utility to parse automatic ids. In order to make it more
extensible I was thinking if use a
"parse
scheme
"
as in the hardwareSpec: different parsers with a Map that relates keys with
parsers, but it seems excessive to me for the moment, considering that only
ram and cores are parsed.
BTW, I'm pushing at this branch:
https://github.com/ivanlomba/jclouds/tree/gsoc2016-JCLOUDS-482
2016-06-15 19:55 GMT+02:00 Ignasi Barrera <na...@apache.org>:
> +1 to use parseable ids.
>
> Regarding the format, let's pick one that allows us to easily extend it in
> the future, in case we need to add there other hardware profile information
> such as disk sizes, etc, but keep it as simple as possible to cover the
> current use case.
>
> I.
> El 13 jun. 2016 1:47 p. m., "Iván Lomba" <iv...@gmail.com> escribió:
>
> > Hi!
> >
> > I'm going to start with the implementation to support cpu and ram.
> > According with discussed here before, I understand that the best approach
> > will be to have auto-generated hardwares, and to take advantage of
> minCores
> > and minRAM to avoid interface changes and casts.
> >
> > Regarding the ids, I'm not sure about what is better or what has more
> > sense: if using randomIds, parseableIds or even combine it? What do you
> > think?
> >
> > Parseable ids that allow to reconstruct the hardware from the id seems to
> > me an useful option. If we decide to use that approach, what format can
> we
> > use? Something like:
> >
> > "automatic-cpu-5-ram-2048..."
> >
> > Regards.
> >
>
Re: JCLOUDS-482 - Add support for arbitrary CPU and RAM in the ComputeService
Posted by Ignasi Barrera <na...@apache.org>.
+1 to use parseable ids.
Regarding the format, let's pick one that allows us to easily extend it in
the future, in case we need to add there other hardware profile information
such as disk sizes, etc, but keep it as simple as possible to cover the
current use case.
I.
El 13 jun. 2016 1:47 p. m., "Iván Lomba" <iv...@gmail.com> escribió:
> Hi!
>
> I'm going to start with the implementation to support cpu and ram.
> According with discussed here before, I understand that the best approach
> will be to have auto-generated hardwares, and to take advantage of minCores
> and minRAM to avoid interface changes and casts.
>
> Regarding the ids, I'm not sure about what is better or what has more
> sense: if using randomIds, parseableIds or even combine it? What do you
> think?
>
> Parseable ids that allow to reconstruct the hardware from the id seems to
> me an useful option. If we decide to use that approach, what format can we
> use? Something like:
>
> "automatic-cpu-5-ram-2048..."
>
> Regards.
>