Re: Rackspace cloudDNS and rate limiting

[Giovanni Toraldo <gi...@cloudesire.com>]

Hi Ignasi,

thanks for your prompt reply, and sorry for my delayed response.

I was hoping to come up with a PR in a few weekends, but I haven't had
any chance to work at it.

It looks easy to replicate the DO behavior in the rackspace driver,
and there is plenty of space to refactor in an unique class to handle
both rackspace and DO.

Finger crossed for me.

Have a nice weekend.


On Wed, Feb 24, 2016 at 4:00 PM, Ignasi Barrera <na...@apache.org> wrote:
> For completeness, this is the commit that added this to DigitalOcean.
> In it you'll see the other referenced classes as long as the tests we
> created:
> https://github.com/jclouds/jclouds/commit/7e866ad6a134357ce34ba7ae5247a088aac5c79e
>
> On 24 February 2016 at 15:53, Ignasi Barrera <na...@apache.org> wrote:
>> Hi Giovanni,
>>
>> Currently there is no explicit retry handler for rate limit errors in
>> Rackspace, but it is something that could be implemented. The
>> DigitalOcean fixes it in quite an elegant way, and I'm pretty sure a
>> similar approach could be aded for Rackspace. I'll try to summarise
>> the relevant bits from the DigitalOcean provider so you can try to
>> follow the same pattern:
>>
>> * First, it declares the DigitalOcean2RateLimitModule [1] that
>> registers a custom error handler for retry limit errors. This module
>> is not configured by default, but users willing to use the automatic
>> rate-limit retry mechanism, can add it to the list of modules when
>> creating the context; that's all it requires from a user's
>> perspective.
>>
>> * Second, the declared RateLimitRetryHandler [2] inspects the request
>> and decides if the error should be propagated or the request should be
>> retried, and when. Note that retry handlers in jclouds work "at HTTP
>> level". That means they are transparent to the provider API method
>> calls. Once call to an interface API method could end up in several
>> HTTP requests or retries, being the rate limit retries some of them.
>> The good thing about this approach is that is transparent to the API
>> and the existing code.
>>
>> * Finally, the DigitalOcean2ErrorHandler [3] propagates a custom
>> rate-limit exception, when the request can't be retried any more, or
>> the user hasn't configured the rate limit module and rate limit errors
>> are just raised. That exception contains detailed information about
>> the rate limit error and allows users to manually handle rate-limit
>> errors, if needed.
>>
>>
>>
>> With these pieces in place it shouldn't be very difficult to implement
>> the same for Rackspace, and I would love to see a pull request for
>> this! If you're up to trying to implement it, I'll be happy to help.
>>
>>
>> I.
>>
>>
>> [1] https://github.com/jclouds/jclouds/blob/master/providers/digitalocean2/src/main/java/org/jclouds/digitalocean2/config/DigitalOcean2RateLimitModule.java
>> [2] https://github.com/jclouds/jclouds/blob/master/providers/digitalocean2/src/main/java/org/jclouds/digitalocean2/handlers/RateLimitRetryHandler.java
>> [3] https://github.com/jclouds/jclouds/blob/master/providers/digitalocean2/src/main/java/org/jclouds/digitalocean2/handlers/DigitalOcean2ErrorHandler.java#L66-L68
>>
>> On 24 February 2016 at 15:35, Giovanni Toraldo
>> <gi...@cloudesire.com> wrote:
>>> Hi guys,
>>>
>>> I've just hit a rate limit error while using rackspace dns service:
>>>
>>> org.jclouds.http.HttpResponseException: command: GET
>>> https://lon.dns.api.rackspacecloud.com/v1.0/10040650/domains/4433011?showRecords=true&showSubdomains=true
>>> HTTP/1.1 failed with response: HTTP/1.1 413 Request Entity Too Large;
>>> content: [{
>>>     "overLimit" : {
>>>         "code" : 413,
>>>         "message" : "OverLimit Retry...",
>>>         "details" : "Error Details...",
>>>    "retryAfter" : "2016-02-24T14:25:57Z"
>>>     }
>>> }]
>>>
>>> full stacktrace https://gist.github.com/gionn/2b32e8734749a71490c3
>>>
>>> Is there an easy way to implement a simple retry feature, maybe using
>>> the suggested retryAfter field of the error response?
>>>
>>> Thanks.
>>>
>>> --
>>> Giovanni Toraldo
>>> Lead Developer
>>> https://cloudesire.com



-- 
Giovanni Toraldo
Lead Developer
https://cloudesire.com