You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@whirr.apache.org by Steve Loughran <st...@hortonworks.com> on 2012/11/14 14:21:29 UTC
understanding rackspace APIs and instances
I'm trying to deploy to rackspace in the UK, not understanding what's going
on. Before anyone says "-user question" -I know, it's just I am trying to
get my service with RPMs installed on the new Rackspace openstack APIs
If someone can help with this I'll assist with updating the docs
as background, the provider list goes:
* Rackspace Cloud Servers UK - tested
Homepage: Optional.of(
http://www.rackspace.co.uk/cloud-hosting/cloud-products/cloud-servers)
Console: Optional.of(https://lon.manage.rackspacecloud.com)
API:
http://docs.rackspacecloud.com/servers/api/v1.0/cs-devguide/content/ch01.html
Configuration options:
whirr.provider = cloudservers-uk
whirr.identity = <Username>
whirr.credential = <API Key>
I take it this is pre-openstack, as it talks of the "first gen api"
There is a listing for the openstack API to the US, but not the UK
* Rackspace Next Generation Cloud Servers US
Homepage: Optional.of(http://www.rackspace.com/cloud/nextgen)
Console: Optional.of(https://mycloud.rackspace.com)
API:
http://docs.rackspace.com/servers/api/v2/cs-devguide/content/ch_preface.html#webhelp-currentid
Configuration options:
whirr.provider = rackspace-cloudservers-us
whirr.identity = <${userName}>
whirr.credential = <${apiKey}>
1. How do I set things up to talk OpenStack APIs to the UK?
2. what does whirr.image do on rackspace infrastructures? It doesn't
seem to pick up a named image of the user.
I'm trying to see where in the source all of this is set up. Is it in
jclouds ?
-steve
Re: understanding rackspace APIs and instances
Posted by Adrian Cole <ad...@jclouds.org>.
P.S.
As you noted in the issue, jclouds isn't currently reading the
retryAfter response. The following issue got distracted with the
overall throttling issue, but started on the topic at hand (ex. when
25 requests are made, jclouds currently breaks instead of sleeping
until next minute)
http://code.google.com/p/jclouds/issues/detail?id=549
probably best to split the throttling stuff in the above issue to
another one, which can address large clusters.
-A
On Wed, Nov 14, 2012 at 2:18 PM, Steve Loughran <st...@hortonworks.com> wrote:
> On 14 November 2012 21:40, Adrian Cole <ad...@jclouds.org> wrote:
>
>> Hi, Steve.
>>
>> The new rackspace cloud has a relatively low request limit per day, so you
>> will need to configure properties to reduce the polling interval.
>>
>>
>> https://github.com/jclouds/jclouds-examples/blob/master/rackspace/src/main/java/org/jclouds/examples/rackspace/cloudservers/CreateServer.java
>>
>>
> I see -is there a way to pass this down through Whirr or shall I sneak
> something in.
>
>
>
>> Wrt creating servers on error response, this should be analyzed and
>> possibly reported to tax.
>>
>> HTH,
>> -A
>>
>
> I'm more worried about the deletion -I think they failed too for the same
> reason, though I don't see why those are limited; the infrastructure could
> just queue them up.
Re: understanding rackspace APIs and instances
Posted by Steve Loughran <st...@hortonworks.com>.
On 14 November 2012 21:40, Adrian Cole <ad...@jclouds.org> wrote:
> Hi, Steve.
>
> The new rackspace cloud has a relatively low request limit per day, so you
> will need to configure properties to reduce the polling interval.
>
>
> https://github.com/jclouds/jclouds-examples/blob/master/rackspace/src/main/java/org/jclouds/examples/rackspace/cloudservers/CreateServer.java
>
>
I see -is there a way to pass this down through Whirr or shall I sneak
something in.
> Wrt creating servers on error response, this should be analyzed and
> possibly reported to tax.
>
> HTH,
> -A
>
I'm more worried about the deletion -I think they failed too for the same
reason, though I don't see why those are limited; the infrastructure could
just queue them up.
Re: understanding rackspace APIs and instances
Posted by Adrian Cole <ad...@jclouds.org>.
Hi, Steve.
The new rackspace cloud has a relatively low request limit per day, so you
will need to configure properties to reduce the polling interval.
https://github.com/jclouds/jclouds-examples/blob/master/rackspace/src/main/java/org/jclouds/examples/rackspace/cloudservers/CreateServer.java
Wrt creating servers on error response, this should be analyzed and
possibly reported to tax.
HTH,
-A
On Nov 14, 2012 11:55 AM, "Steve Loughran" <st...@hortonworks.com> wrote:
> I've got rackspace-cloudservers-uk working for a single node cluster, but
> when I ask for two worker machines, I see stack traces pointing to a lump
> of JSON coming back
>
> org.jclouds.rest.InsufficientResourcesException: {
> "overLimit" : {
> "code" : 413,
> "message" : "OverLimit Retry...",
> "details" : "Error Details...",
> "retryAt" : "2012-11-14T19:34:04UTC"
> }
> }
>
> This is clearly some throttling thing -either rate of RESTy requests or
> rate of API calls in general -I'll have to understand that.
>
> When the request fails, cluster launch is considered failed, so
> the /Users/stevel/.whirr/hadoop/instances file doesn't get created
>
> The machines are there though:
> $ nova list
>
> +--------------------------------------+------------+--------+-----------------------------------------------------------------------------------+
> | ID | Name | Status | Networks
> |
>
> +--------------------------------------+------------+--------+-----------------------------------------------------------------------------------+
> | e13cf52f-4290-4b4a-92e1-24f9922a4209 | hadoop-1f2 | ACTIVE |
> public=5.79.7.206, 2a00:1a48:7803:0107:176c:89a0:ff08:2211;
> private=10.177.196.41 |
> | 1eb6df60-d55a-4c40-9279-f2dbd396555c | hadoop-351 | ACTIVE |
> public=2a00:1a48:7803:0107:176c:89a0:ff08:220f, 5.79.7.85;
> private=10.177.195.157 |
> | 08bbfbcb-e37c-4335-b2d1-4d99e6083ab1 | hadoop-b19 | ACTIVE |
> public=2a00:1a48:7803:0107:176c:89a0:ff08:220e, 5.79.7.90;
> private=10.177.197.33 |
>
> +--------------------------------------+------------+--------+-----------------------------------------------------------------------------------+
>
> this means that the request did go through eventually, but wasn't picked up
> -so whirr is leaking vm instances
>
Re: understanding rackspace APIs and instances
Posted by Steve Loughran <st...@hortonworks.com>.
I've got rackspace-cloudservers-uk working for a single node cluster, but
when I ask for two worker machines, I see stack traces pointing to a lump
of JSON coming back
org.jclouds.rest.InsufficientResourcesException: {
"overLimit" : {
"code" : 413,
"message" : "OverLimit Retry...",
"details" : "Error Details...",
"retryAt" : "2012-11-14T19:34:04UTC"
}
}
This is clearly some throttling thing -either rate of RESTy requests or
rate of API calls in general -I'll have to understand that.
When the request fails, cluster launch is considered failed, so
the /Users/stevel/.whirr/hadoop/instances file doesn't get created
The machines are there though:
$ nova list
+--------------------------------------+------------+--------+-----------------------------------------------------------------------------------+
| ID | Name | Status | Networks
|
+--------------------------------------+------------+--------+-----------------------------------------------------------------------------------+
| e13cf52f-4290-4b4a-92e1-24f9922a4209 | hadoop-1f2 | ACTIVE |
public=5.79.7.206, 2a00:1a48:7803:0107:176c:89a0:ff08:2211;
private=10.177.196.41 |
| 1eb6df60-d55a-4c40-9279-f2dbd396555c | hadoop-351 | ACTIVE |
public=2a00:1a48:7803:0107:176c:89a0:ff08:220f, 5.79.7.85;
private=10.177.195.157 |
| 08bbfbcb-e37c-4335-b2d1-4d99e6083ab1 | hadoop-b19 | ACTIVE |
public=2a00:1a48:7803:0107:176c:89a0:ff08:220e, 5.79.7.90;
private=10.177.197.33 |
+--------------------------------------+------------+--------+-----------------------------------------------------------------------------------+
this means that the request did go through eventually, but wasn't picked up
-so whirr is leaking vm instances
Re: understanding rackspace APIs and instances
Posted by Steve Loughran <st...@hortonworks.com>.
On 14 November 2012 16:02, Adrian Cole <ad...@jclouds.org> wrote:
> Hi, Steve.
>
> jclouds 1.5.2 missed the pom definition for rackspace-cloudservers-u[sk].
> This should be fixed in 1.5.3, which is available on maven central as of
> this morning.
>
yes, that does it:
* Rackspace Next Generation Cloud Servers UK
Homepage: Optional.of(http://www.rackspace.co.uk/opencloud)
Console: Optional.of(https://mycloud.rackspace.co.uk/)
API:
http://docs.rackspace.com/servers/api/v2/cs-devguide/content/ch_preface.html#webhelp-currentid
Configuration options:
whirr.provider = rackspace-cloudservers-uk
whirr.identity = <${userName}>
whirr.credential = <${apiKey}>
>
> For images, the naming convention in nova installs like next-gen is
> zoneid/imageid
>
OK -I'll just have to work out that little detail from rackspace...
Re: understanding rackspace APIs and instances
Posted by Adrian Cole <ad...@jclouds.org>.
Hi, Steve.
jclouds 1.5.2 missed the pom definition for rackspace-cloudservers-u[sk].
This should be fixed in 1.5.3, which is available on maven central as of
this morning.
For images, the naming convention in nova installs like next-gen is
zoneid/imageid
HTH,
-A
On Nov 14, 2012 5:21 AM, "Steve Loughran" <st...@hortonworks.com> wrote:
> I'm trying to deploy to rackspace in the UK, not understanding what's going
> on. Before anyone says "-user question" -I know, it's just I am trying to
> get my service with RPMs installed on the new Rackspace openstack APIs
>
> If someone can help with this I'll assist with updating the docs
>
>
> as background, the provider list goes:
>
> * Rackspace Cloud Servers UK - tested
> Homepage: Optional.of(
> http://www.rackspace.co.uk/cloud-hosting/cloud-products/cloud-servers)
> Console: Optional.of(https://lon.manage.rackspacecloud.com)
> API:
>
> http://docs.rackspacecloud.com/servers/api/v1.0/cs-devguide/content/ch01.html
> Configuration options:
> whirr.provider = cloudservers-uk
> whirr.identity = <Username>
> whirr.credential = <API Key>
>
> I take it this is pre-openstack, as it talks of the "first gen api"
>
> There is a listing for the openstack API to the US, but not the UK
> * Rackspace Next Generation Cloud Servers US
> Homepage: Optional.of(http://www.rackspace.com/cloud/nextgen)
> Console: Optional.of(https://mycloud.rackspace.com)
> API:
>
> http://docs.rackspace.com/servers/api/v2/cs-devguide/content/ch_preface.html#webhelp-currentid
> Configuration options:
> whirr.provider = rackspace-cloudservers-us
> whirr.identity = <${userName}>
> whirr.credential = <${apiKey}>
>
>
> 1. How do I set things up to talk OpenStack APIs to the UK?
> 2. what does whirr.image do on rackspace infrastructures? It doesn't
> seem to pick up a named image of the user.
>
> I'm trying to see where in the source all of this is set up. Is it in
> jclouds ?
>
> -steve
>
Re: understanding rackspace APIs and instances
Posted by Steve Loughran <st...@hortonworks.com>.
I should add that jclouds 1.5.2 doesn't handle this
Caused by: java.util.NoSuchElementException: key
[rackspace-cloudservers-uk] not in the list of providers or apis:
{providers=[aws-ec2, aws-s3, azureblob, bluelock-vcloud-zone01,
cloudfiles-uk, cloudfiles-us, cloudonestorage, cloudservers-uk,
cloudservers-us, cloudsigma-lvs, cloudsigma-zrh, elastichosts-lon-b,
elastichosts-lon-p, elastichosts-sat-p, eucalyptus-partnercloud-ec2,
eucalyptus-partnercloud-s3, go2cloud-jhb1, gogrid,
greenhousedata-element-vcloud, hpcloud-compute, hpcloud-objectstorage,
ninefold-compute, ninefold-storage, openhosting-east1,
rackspace-cloudservers-us, rimuhosting, serverlove-z1-man,
skalicloud-sdg-my, slicehost, softlayer, stratogen-vcloud-mycloud,
synaptic-storage, trmk-ecloud, trmk-vcloudexpress, trystack-nova],
apis=[atmos, byon, cloudfiles, cloudservers, cloudsigma, cloudstack,
deltacloud, ec2, elasticstack, eucalyptus, filesystem, transient, stub,
nova, openstack-keystone, openstack-nova, openstack-nova-ec2,
rackspace-cloudidentity, s3, swift, swift-keystone, vcloud, walrus]}
at org.jclouds.ContextBuilder.newBuilder(ContextBuilder.java:130)
at org.apache.whirr.service.ComputeCache$1.load(ComputeCache.java:94)
at org.apache.whirr.service.ComputeCache$1.load(ComputeCache.java:89)
at
com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3589)
at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2374)
at
com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2337)
at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2252)
... 13 more
even though the 1.5 docs say that it is a new platform. Only the -us one is
showing up.
http://www.jclouds.org/documentation/releasenotes/1.5/
On 14 November 2012 13:21, Steve Loughran <st...@hortonworks.com> wrote:
>
>
> I'm trying to deploy to rackspace in the UK, not understanding what's
> going on. Before anyone says "-user question" -I know, it's just I am
> trying to get my service with RPMs installed on the new Rackspace openstack
> APIs
>
> If someone can help with this I'll assist with updating the docs
>
>
> as background, the provider list goes:
>
> * Rackspace Cloud Servers UK - tested
> Homepage: Optional.of(
> http://www.rackspace.co.uk/cloud-hosting/cloud-products/cloud-servers)
> Console: Optional.of(https://lon.manage.rackspacecloud.com)
> API:
> http://docs.rackspacecloud.com/servers/api/v1.0/cs-devguide/content/ch01.html
> Configuration options:
> whirr.provider = cloudservers-uk
> whirr.identity = <Username>
> whirr.credential = <API Key>
>
> I take it this is pre-openstack, as it talks of the "first gen api"
>
> There is a listing for the openstack API to the US, but not the UK
> * Rackspace Next Generation Cloud Servers US
> Homepage: Optional.of(http://www.rackspace.com/cloud/nextgen)
> Console: Optional.of(https://mycloud.rackspace.com)
> API:
> http://docs.rackspace.com/servers/api/v2/cs-devguide/content/ch_preface.html#webhelp-currentid
> Configuration options:
> whirr.provider = rackspace-cloudservers-us
> whirr.identity = <${userName}>
> whirr.credential = <${apiKey}>
>
>
> 1. How do I set things up to talk OpenStack APIs to the UK?
> 2. what does whirr.image do on rackspace infrastructures? It doesn't
> seem to pick up a named image of the user.
>
> I'm trying to see where in the source all of this is set up. Is it in
> jclouds ?
>
> -steve
>
>