You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@jclouds.apache.org by Iván Lomba <no...@github.com> on 2016/08/14 22:15:41 UTC
[jclouds/jclouds-site] Arbitrary CPU and RAM supported in
ComputeService (#184)
Summary of new feature, the post includes:
- Description
- Providers supported
- How to create custom hardwares
- Further development
You can view, comment on, or merge this pull request online at:
https://github.com/jclouds/jclouds-site/pull/184
-- Commit Summary --
* first draft of arbitraryCpuRam post
* Add some links, how to configure it, improve examples
* polish the post a bit
* fix misprint
* Add ProfitBricks examples and Further development
-- File Changes --
A _posts/2016-08-15-arbitrary-cpu-ram.md (91)
-- Patch Links --
https://github.com/jclouds/jclouds-site/pull/184.patch
https://github.com/jclouds/jclouds-site/pull/184.diff
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds-site/pull/184
Re: [jclouds/jclouds-site] Blog post: Arbitrary CPU and RAM supported
in ComputeService (#184)
Posted by Ignasi Barrera <no...@github.com>.
> +
> +### Providers supported
> +There are several providers that support arbitrary values of CPU and RAM like Docker, ElasticHosts, Google Compute Engine, etc. The first available providers supported by the new feature are:
> +
> +* [Google Compute Engine](https://cloud.google.com/compute/)
> +* [ProfitBricks](https://www.profitbricks.com/)
> +
> +To configure the new feature in other providers add a bind() to the ArbitraryCpuRamTemplateBuilderImpl class at the provider's context module:
> +
> +{% highlight Java %}
> +bind(TemplateBuilderImpl.class).to(ArbitraryCpuRamTemplateBuilderImpl.class);
> +{% endhighlight %}
> +
> +Also is necessary to modify the function that transform a node from the provider model to the portable model of jclouds, to include the new automatic hardwareId (if apply).
> +
> +### How to create custom hardwares
Don't say "create custom hardwares", as we're not creating anything. Better refer to the feature as "using custom hardwares".
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds-site/pull/184/files/aca97ff41fd96dee3312ff6b6fef0a35d005df77#r74839498
Re: [jclouds/jclouds-site] Blog post: Arbitrary CPU and RAM supported
in ComputeService (#184)
Posted by Ignasi Barrera <no...@github.com>.
It really depends on the use case. Users using the *minCores/Ram* methods don't really care about which concrete hardware profile is used as long as it satisfies those minimums. The current approach respects that: it tries to find an existing profile that meets the criteria and, if none is found, returns an *automatic* one with the given values.
Users using the *hardwareId* method want to use concrete profiles. Since automatic profiles don't exist but they need an ID in order to honor the Hardware and NodeMetadata interfaces, we use a parseable ID that can be used in the `TemplateBuilder.hardwareId` method too. This allows users to create deterministic code for automatic/explicit hardwares the same way they did before.
In the end it is about the use case, which shouldn't have changed with the introduction of these feature. The reasoning behind the choice of one or the other should remain the same.
Finally, regarding the first option (*minCores/Ram*), now providers will probably always find an existing hardware (and not fallback to the automatic behaviour). This is because in providers that don't really have hardware profiles we were returning mock values in the `ComputeService.listHardwares` method. Those should be changed to return an empty list, thus better leveraging the automatic hardware feature, but we've keeped them in this first version for backward compatibility (ther might be users coding against those mock hardwares).
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds-site/pull/184#issuecomment-241263446
Re: [jclouds/jclouds-site] Blog post: Arbitrary CPU and RAM supported
in ComputeService (#184)
Posted by Andrew Phillips <no...@github.com>.
Thanks for all the changes, @ivanlomba! There are a bunch of small language things that I'm happy to do as a final pass, but from a content perspective I think we're pretty much there.
The one question I still have is whether we can give any kind of guidance about whether to use the builder or to construct an appropriate `automatic:...` ID string. We still mention both and give examples of both without recommending one or the other (whether in general or in particular cases).
If we feel they are really totally equivalent, perhaps point out that using the builder makes you slightly less prone to problems with string manipulation that you get when you construct the ID string yourself? Also, are there any helpers/utility objects (even if it's just something like `UriBuilder..addParam(...).addParam(...)...getQueryString()`) that can make constructing that string easier for users?
/cc @nacx
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds-site/pull/184#issuecomment-241214668
Re: [jclouds/jclouds-site] Blog post: Arbitrary CPU and RAM supported
in ComputeService (#184)
Posted by Ignasi Barrera <no...@github.com>.
Closed #184.
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds-site/pull/184#event-762611567
Re: [jclouds/jclouds-site] Blog post: Arbitrary CPU and RAM supported
in ComputeService (#184)
Posted by Iván Lomba <no...@github.com>.
@ivanlomba pushed 1 commit.
fbd1a13 Update according PR comments
--
You are receiving this because you are subscribed to this thread.
View it on GitHub:
https://github.com/jclouds/jclouds-site/pull/184/files/aca97ff41fd96dee3312ff6b6fef0a35d005df77..fbd1a1349565bdd2ea92ce8a22e894d7a635cc3b
Re: [jclouds/jclouds-site] Blog post: Arbitrary CPU and RAM supported
in ComputeService (#184)
Posted by jclouds-commentator <no...@github.com>.
to review your changes.
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds-site/pull/184#issuecomment-239747496
Re: [jclouds/jclouds-site] Blog post: Arbitrary CPU and RAM supported
in ComputeService (#184)
Posted by Iván Lomba <no...@github.com>.
@ivanlomba pushed 1 commit.
e4b191c Add automaticHardwareIdSpecBuilder example
--
You are receiving this because you are subscribed to this thread.
View it on GitHub:
https://github.com/jclouds/jclouds-site/pull/184/files/9d99667d1a3b03f895d18eb5f9c8c4efc6844ea0..e4b191c690f6e88dd6087a00f03e68764871350b
Re: [jclouds/jclouds-site] Blog post: Arbitrary CPU and RAM supported
in ComputeService (#184)
Posted by Iván Lomba <no...@github.com>.
@ivanlomba pushed 1 commit.
9d99667 Add hardware profiles introduction and change order
--
You are receiving this because you are subscribed to this thread.
View it on GitHub:
https://github.com/jclouds/jclouds-site/pull/184/files/fbd1a1349565bdd2ea92ce8a22e894d7a635cc3b..9d99667d1a3b03f895d18eb5f9c8c4efc6844ea0
Re: [jclouds/jclouds-site] Blog post: Arbitrary CPU and RAM supported
in ComputeService (#184)
Posted by jclouds-commentator <no...@github.com>.
to review your changes.
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds-site/pull/184#issuecomment-241261969
Re: [jclouds/jclouds-site] Blog post: Arbitrary CPU and RAM supported
in ComputeService (#184)
Posted by Iván Lomba <no...@github.com>.
@ivanlomba pushed 1 commit.
91db599 Fix minDisk setter on templateBuilder example
--
You are receiving this because you are subscribed to this thread.
View it on GitHub:
https://github.com/jclouds/jclouds-site/pull/184/files/fb603652aef4ca8e04beadfec6198a665b434419..91db599208431d6bb9880c6e930c76b1ec9ee7e1
Re: [jclouds/jclouds-site] Blog post: Arbitrary CPU and RAM supported
in ComputeService (#184)
Posted by Ignasi Barrera <no...@github.com>.
Thanks @ivanlomba!
@demobox Would you have some time to have a look at this before we publish it?
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds-site/pull/184#issuecomment-239936787
Re: [jclouds/jclouds-site] Blog post: Arbitrary CPU and RAM supported
in ComputeService (#184)
Posted by Ignasi Barrera <no...@github.com>.
Squashed, pushed to master as [386c1763](http://git-wip-us.apache.org/repos/asf/jclouds-site/commit/386c1763) and published!
@ivanlomba feel free to include the link in your deliverables as long as your commits to this repo and please, open a PR to include the last comments from @demobox.
I'm closing this PR now, but feel free to keep the discussion here if appropriate.
Thanks both!
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds-site/pull/184#issuecomment-241331495
Re: [jclouds/jclouds-site] Blog post: Arbitrary CPU and RAM supported
in ComputeService (#184)
Posted by jclouds-commentator <no...@github.com>.
to review your changes.
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds-site/pull/184#issuecomment-239701371
Re: [jclouds/jclouds-site] Blog post: Arbitrary CPU and RAM supported
in ComputeService (#184)
Posted by jclouds-commentator <no...@github.com>.
to review your changes.
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds-site/pull/184#issuecomment-240156331
Re: [jclouds/jclouds-site] Blog post: Arbitrary CPU and RAM supported
in ComputeService (#184)
Posted by Iván Lomba <no...@github.com>.
@ivanlomba pushed 1 commit.
aca97ff Fix disk explanation
--
You are receiving this because you are subscribed to this thread.
View it on GitHub:
https://github.com/jclouds/jclouds-site/pull/184/files/91db599208431d6bb9880c6e930c76b1ec9ee7e1..aca97ff41fd96dee3312ff6b6fef0a35d005df77
Re: [jclouds/jclouds-site] Blog post: Arbitrary CPU and RAM supported
in ComputeService (#184)
Posted by Ignasi Barrera <no...@github.com>.
I've just changed the date of the post. Please make sure you update the URL! It is the one for August 22.
@demobox no need to hurry for this at all; once amended we'll tweet the post from our Twitter account.
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds-site/pull/184#issuecomment-241333939
Re: [jclouds/jclouds-site] Blog post: Arbitrary CPU and RAM supported
in ComputeService (#184)
Posted by Ignasi Barrera <no...@github.com>.
The previous comment is incomplete because the site CI build failed. The error is:
HTTP/1.1 403 Forbidden; content: [{"userDisabled":{"code":403,"message":"User 'apachejclouds' is disabled."}}
@zack-shoylev could yoi have a look?
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds-site/pull/184#issuecomment-239702279
Re: [jclouds/jclouds-site] Blog post: Arbitrary CPU and RAM supported
in ComputeService (#184)
Posted by Andrew Phillips <no...@github.com>.
> + .build();
> +compute.createNodesInGroup("jclouds", 1, template);
> +{% endhighlight %}
> +
> +As some providers support also hardware profiles, and using them provides hardware optimization or in some cases cheaper pricings, when a provider support both, Template Builder first will look if some hardware profile matches the specified values.
> +
> +### Further development
> +
> +* **Support other providers**: add support for other providers such as [ElasticHosts](https://www.elastichosts.com/) and [Docker](https://www.docker.com/).
> +* **Improve AutomaticHardwareSpec**: improve the AutomaticHardwareSpec with specific parsers for every parameter in order to support more custom parameters and some fields, like bootDisk and durable (part of volumes), that are currently hardcoded to true.
> +* **Usage examples of the new features**: create examples of the new features in the jclouds-examples repo.
> +
> +
> +### Special thanks
> +
> +Special thanks to [Ignasi Barrera](https://github.com/nacx) for all the help, [Andrew Phillips](https://github.com/demobox) for code reviews and the rest of jclouds comunity.
Thanks, Ivan, but I think Ignasi should get the credit here - he did **much** more work than I did ;-)
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds-site/pull/184/files/aca97ff41fd96dee3312ff6b6fef0a35d005df77#r75134296
Re: [jclouds/jclouds-site] Blog post: Arbitrary CPU and RAM supported
in ComputeService (#184)
Posted by Andrew Phillips <no...@github.com>.
Thanks for putting this together, @ivanlomba! Some general questions/comments:
* Who is the intended target audience here: people who want to _use_ these providers or people who want to _implement_ them? Or both? At the moment, we are mixing both types of information, and if we want to do that I think we should focus on the "use arbitrary CPU and RAM" first, since that's likely to be much more common.
* When we demonstrate how to use it, is there some guidance we can give about which mechanism (hardware ID vs. template builder) is preferred/recommended? Users generally find it easier to be given _one_ option, rather than multiple, especially if there isn't clear information about when to choose which option.
* The example of some providers requiring disk info: is there any way for a user to know that? Or do they have to try without and wait for an error message?
* We introduce "hardware profiles" as terms quite early on without explaining them - are we assuming they are sufficiently common that everyone (or, at least, all the likely readers) will know what they are?
/cc @nacx
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds-site/pull/184#issuecomment-240432085
Re: [jclouds/jclouds-site] Blog post: Arbitrary CPU and RAM supported
in ComputeService (#184)
Posted by Ignasi Barrera <no...@github.com>.
> +slug: arbitrary-cpu-ram
> +title: Arbitrary CPU and RAM supported in the ComputeService
> +---
> +
> +As part of a [Google Summer of Code](https://developers.google.com/open-source/gsoc/) project has been added a feature to allow users to set manually specific values of CPU and RAM.
> +<!--more-->
> +
> +The previous Compute Service abstraction assumed that all providers had hardware profiles, but some providers such as [ProfitBricks](https://www.profitbricks.com/) or [ElasticHosts](https://www.elastichosts.com/) do not have the hardware profiles concept and the previous implementation provides a fixed configuration with a fixed (hardcoded) list just to conform the interface. The new implementation allows to use custom hardwares or both (when supported) hardware profiles and custom hardwares.
> +
> +### Providers supported
> +There are several providers that support arbitrary values of CPU and RAM like Docker, ElasticHosts, Google Compute Engine, etc. The first available providers supported by the new feature are:
> +
> +* [Google Compute Engine](https://cloud.google.com/compute/)
> +* [ProfitBricks](https://www.profitbricks.com/)
> +
> +To configure the new feature in other providers add a bind() to the ArbitraryCpuRamTemplateBuilderImpl class at the provider's context module:
Change the `ArbitraryCpuRamTemplateBuilderImpl `, to highlight it as code.
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds-site/pull/184/files/aca97ff41fd96dee3312ff6b6fef0a35d005df77#r74839320
Re: [jclouds/jclouds-site] Blog post: Arbitrary CPU and RAM supported
in ComputeService (#184)
Posted by jclouds-commentator <no...@github.com>.
to review your changes.
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds-site/pull/184#issuecomment-241190345
Re: [jclouds/jclouds-site] Blog post: Arbitrary CPU and RAM supported
in ComputeService (#184)
Posted by Iván Lomba <no...@github.com>.
Thanks @demobox
>
Who is the intended target audience here: people who want to use these providers or people who want to implement them? Or both? At the moment, we are mixing both types of information, and if we want to do that I think we should focus on the "use arbitrary CPU and RAM" first, since that's likely to be much more common.
The main intention of the post is to give a general idea of the new feature and future work, specially how to use it, but also introducing how to extend it to other providers and commenting next steps. To avoid mixing information, I will put first the user guide and below the developers guide.
> When we demonstrate how to use it, is there some guidance we can give about which mechanism (hardware ID vs. template builder) is preferred/recommended? Users generally find it easier to be given one option, rather than multiple, especially if there isn't clear information about when to choose which option.
I think maybe explain a little bit the two options(clarifying the implications of using the hardwareId or "mins" option) can be better that only give a recommended one:
1. AutomaticId: the template builder checks if a hardware profile with exactly that
hardwareId exist, if don't use the automatic.
2. Mins: the template builder checks if a hardware profile match with the provided minRam and minCores, if not use the automatic.
> The example of some providers requiring disk info: is there any way for a user to know that? Or do they have to try without and wait for an error message?
They have to try without know that.
> We introduce "hardware profiles" as terms quite early on without explaining them - are we assuming they are sufficiently common that everyone (or, at least, all the likely readers) will know what they are?
I think that a potential user of jclouds will understand it, but I can introduce the hardware profiles if you think that is necessary.
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds-site/pull/184#issuecomment-240995089
Re: [jclouds/jclouds-site] Blog post: Arbitrary CPU and RAM supported
in ComputeService (#184)
Posted by Ignasi Barrera <no...@github.com>.
> +{% highlight Java %}
> +Template template = templateBuilder
> + .minCores(2)
> + .minRam(4096)
> + .minDisk(100)
> + .build();
> +compute.createNodesInGroup("jclouds", 1, template);
> +{% endhighlight %}
> +
> +As some providers support also hardware profiles, and using them provides hardware optimization or in some cases cheaper pricings, when a provider support both, Template Builder first will look if some hardware profile matches the specified values.
> +
> +### Further development
> +
> +* **Support other providers**: add support for other providers such as [ElasticHosts](https://www.elastichosts.com/) and [Docker](https://www.docker.com/).
> +* **Improve AutomaticHardwareSpec**: improve the AutomaticHardwareSpec with specific parsers for every parameter in order to support more custom parameters and some fields, like bootDisk and durable (part of volumes), that are currently hardcoded to true.
> +* **Usage examples of the new features**: create examples of the new features in the jclouds-examples repo.
Add ti this list the remaining things for ProfitBricks:
* Add a custom implementation of the TemplateBuilderImpl to fail early when users don't set the minDisk.
* That custom implementation should also accept hardware IDs with the "automatic" prefix.
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds-site/pull/184/files/aca97ff41fd96dee3312ff6b6fef0a35d005df77#r74839833
Re: [jclouds/jclouds-site] Blog post: Arbitrary CPU and RAM supported
in ComputeService (#184)
Posted by Andrew Phillips <no...@github.com>.
> are you OK with merging this early (given that the content is OK) for this purpose, even if you amend the syntax once publised?
@nacx Absolutely! Sorry for the late response - only just seeing this :-(
> Users using the hardwareId method want to use concrete profiles. Since automatic profiles don't exist but they need an ID in order to honor the Hardware and NodeMetadata interfaces, we use a parseable ID that can be used in the TemplateBuilder.hardwareId method too.
Ah, thanks for clarifying. In that case, if I'm understanding the correctly the _only_ use case for a user _explicitly_ setting an `automatic:` ID would be if there's a matching hardware profile but the user does **not** want to use it, for some reason?
If there is **no** matching hardware profile, both methods will return the same for a provider that supports automatic profiles - is that correct? If so, I'd be tempted to recommend the `.minCores` method generally and point out (if not already done) that this _could_ cause the implementation to return a matching hardware profile if one happens to match those specs. For users that explicitly do **not** want an existing profile, we can recommend that they set the ID explicitly.
This, I think, comes close to respecting the origin of the ID as essentially being something we need to generate for consistency with the rest of the system. Would be curious to hear your thoughts on that, of course...but _after_ this is published, most likely ;-)
/cc @nacx @ivanlomba
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds-site/pull/184#issuecomment-241290018
Re: [jclouds/jclouds-site] Blog post: Arbitrary CPU and RAM supported
in ComputeService (#184)
Posted by jclouds-commentator <no...@github.com>.
to review your changes.
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds-site/pull/184#issuecomment-239765205
Re: [jclouds/jclouds-site] Blog post: Arbitrary CPU and RAM supported
in ComputeService (#184)
Posted by Iván Lomba <no...@github.com>.
@nacex I added a new example using the AutomaticHardwareIdSpecBuilder.
@demobox I'm not sure about which option can be the best recommendation: one always creates a custom hardware (if user specify hardwareId) and the other first search for hardware profiles with similar values (minRam, minCores). For that reason I told you in my previous comment that I think that a better option was clarify the implications of any method.
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds-site/pull/184#issuecomment-241261820
Re: [jclouds/jclouds-site] Blog post: Arbitrary CPU and RAM supported
in ComputeService (#184)
Posted by Ignasi Barrera <no...@github.com>.
Thanks for reviewing @demobox!
@ivanlomba could you add some examples of using the existing utility class to build the automatic id string, as suggested?
If you add the change **today** and @demobox agrees, I'll merge the PR and publish the post. This would allow you to have a URL to add to the GSoC deliverables. @demobox are you OK with merging this early (given that the content is OK) for this purpose, even if you amend the syntax once publised?
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds-site/pull/184#issuecomment-241260104