You are viewing a plain text version of this content. The canonical link for it is here.
Posted to httpclient-users@hc.apache.org by Mike Conley <mi...@unizin.org> on 2015/10/14 22:49:11 UTC
Responding adaptively to 503 errors (including respecting the Retry-After header)
Hi.
I’m writing an OAI harvester (it’s an obscure protocol you’ve probably never heard of for the exchange of metadata with content repositories). The implementation guidelines mention 503 handling: http://www.openarchives.org/OAI/2.0/guidelines-harvester.htm#FlowControl <http://www.openarchives.org/OAI/2.0/guidelines-harvester.htm#FlowControl>. In particular, the suggestion is to respect the Retry-After header if present.
When last I did this, I had to implement it myself, but I see that HttpClient 4 has 503 handling built in.
http://hc.apache.org/httpcomponents-client-ga/httpclient/apidocs/org/apache/http/client/ServiceUnavailableRetryStrategy.html <http://hc.apache.org/httpcomponents-client-ga/httpclient/apidocs/org/apache/http/client/ServiceUnavailableRetryStrategy.html>
But I’m not sure how to use this if the retry interval should vary per-response. getRetryInterval takes no arguments, and in particular has no access to the HttpResponse, so I’m not sure how I could go about using this. I had considered storing something when the call to retryRequest is made, but I’m not sure this is safe to do (it seems likely that it is not, since the strategy is almost certainly shared among multiple threads). It also seems kind of kludgy to me. Is there a way to do this cleanly? If not, is there a thread safe way to keep this state around between calls to the two methods?
Re: Responding adaptively to 503 errors (including respecting the
Retry-After header)
Posted by Stefan Magnus Landrø <st...@gmail.com>.
It was added in this commit:
https://github.com/apache/httpclient/commit/d5c520a8e1231ae60142158781f5c9bf978fd8e9
2015-10-14 23:02 GMT+02:00 Mike Conley <mi...@unizin.org>:
> Oh, nice! I assume this hasn’t been released yet, though, right? If it has
> been, the javadocs need updating :P
>
> Glad it’s been handled either way. Thanks for the info.
>
> > On Oct 14, 2015, at 4:59 PM, Stefan Magnus Landrø <
> stefan.landro@gmail.com> wrote:
> >
> > Newer versions handle the retry-after header:
> >
> https://github.com/apache/httpclient/blob/trunk/httpclient/src/main/java/org/apache/http/client/ServiceUnavailableRetryStrategy.java
> >
> > Have a look at the default impl:
> >
> >
> https://github.com/apache/httpclient/blob/trunk/httpclient/src/main/java/org/apache/http/impl/client/DefaultServiceUnavailableRetryStrategy.java
> >
> > 2015-10-14 22:49 GMT+02:00 Mike Conley <mi...@unizin.org>:
> >
> >> Hi.
> >>
> >> I’m writing an OAI harvester (it’s an obscure protocol you’ve probably
> >> never heard of for the exchange of metadata with content repositories).
> The
> >> implementation guidelines mention 503 handling:
> >>
> http://www.openarchives.org/OAI/2.0/guidelines-harvester.htm#FlowControl.
> >> In particular, the suggestion is to respect the Retry-After header if
> >> present.
> >>
> >> When last I did this, I had to implement it myself, but I see that
> >> HttpClient 4 has 503 handling built in.
> >>
> >>
> >>
> http://hc.apache.org/httpcomponents-client-ga/httpclient/apidocs/org/apache/http/client/ServiceUnavailableRetryStrategy.html
> >>
> >> But I’m not sure how to use this if the retry interval should vary
> >> per-response. getRetryInterval takes no arguments, and in particular
> has no
> >> access to the HttpResponse, so I’m not sure how I could go about using
> >> this. I had considered storing something when the call to retryRequest
> is
> >> made, but I’m not sure this is safe to do (it seems likely that it is
> not,
> >> since the strategy is almost certainly shared among multiple threads).
> It
> >> also seems kind of kludgy to me. Is there a way to do this cleanly? If
> not,
> >> is there a thread safe way to keep this state around between calls to
> the
> >> two methods?
> >>
> >>
> >
> >
> > --
> > BEKK Open
> > http://open.bekk.no
> >
> > TesTcl - a unit test framework for iRules
> > http://testcl.com
>
>
--
BEKK Open
http://open.bekk.no
TesTcl - a unit test framework for iRules
http://testcl.com
Re: Responding adaptively to 503 errors (including respecting the Retry-After header)
Posted by Mike Conley <mi...@unizin.org>.
Oh, nice! I assume this hasn’t been released yet, though, right? If it has been, the javadocs need updating :P
Glad it’s been handled either way. Thanks for the info.
> On Oct 14, 2015, at 4:59 PM, Stefan Magnus Landrø <st...@gmail.com> wrote:
>
> Newer versions handle the retry-after header:
> https://github.com/apache/httpclient/blob/trunk/httpclient/src/main/java/org/apache/http/client/ServiceUnavailableRetryStrategy.java
>
> Have a look at the default impl:
>
> https://github.com/apache/httpclient/blob/trunk/httpclient/src/main/java/org/apache/http/impl/client/DefaultServiceUnavailableRetryStrategy.java
>
> 2015-10-14 22:49 GMT+02:00 Mike Conley <mi...@unizin.org>:
>
>> Hi.
>>
>> I’m writing an OAI harvester (it’s an obscure protocol you’ve probably
>> never heard of for the exchange of metadata with content repositories). The
>> implementation guidelines mention 503 handling:
>> http://www.openarchives.org/OAI/2.0/guidelines-harvester.htm#FlowControl.
>> In particular, the suggestion is to respect the Retry-After header if
>> present.
>>
>> When last I did this, I had to implement it myself, but I see that
>> HttpClient 4 has 503 handling built in.
>>
>>
>> http://hc.apache.org/httpcomponents-client-ga/httpclient/apidocs/org/apache/http/client/ServiceUnavailableRetryStrategy.html
>>
>> But I’m not sure how to use this if the retry interval should vary
>> per-response. getRetryInterval takes no arguments, and in particular has no
>> access to the HttpResponse, so I’m not sure how I could go about using
>> this. I had considered storing something when the call to retryRequest is
>> made, but I’m not sure this is safe to do (it seems likely that it is not,
>> since the strategy is almost certainly shared among multiple threads). It
>> also seems kind of kludgy to me. Is there a way to do this cleanly? If not,
>> is there a thread safe way to keep this state around between calls to the
>> two methods?
>>
>>
>
>
> --
> BEKK Open
> http://open.bekk.no
>
> TesTcl - a unit test framework for iRules
> http://testcl.com
Re: Responding adaptively to 503 errors (including respecting the
Retry-After header)
Posted by Stefan Magnus Landrø <st...@gmail.com>.
Newer versions handle the retry-after header:
https://github.com/apache/httpclient/blob/trunk/httpclient/src/main/java/org/apache/http/client/ServiceUnavailableRetryStrategy.java
Have a look at the default impl:
https://github.com/apache/httpclient/blob/trunk/httpclient/src/main/java/org/apache/http/impl/client/DefaultServiceUnavailableRetryStrategy.java
2015-10-14 22:49 GMT+02:00 Mike Conley <mi...@unizin.org>:
> Hi.
>
> I’m writing an OAI harvester (it’s an obscure protocol you’ve probably
> never heard of for the exchange of metadata with content repositories). The
> implementation guidelines mention 503 handling:
> http://www.openarchives.org/OAI/2.0/guidelines-harvester.htm#FlowControl.
> In particular, the suggestion is to respect the Retry-After header if
> present.
>
> When last I did this, I had to implement it myself, but I see that
> HttpClient 4 has 503 handling built in.
>
>
> http://hc.apache.org/httpcomponents-client-ga/httpclient/apidocs/org/apache/http/client/ServiceUnavailableRetryStrategy.html
>
> But I’m not sure how to use this if the retry interval should vary
> per-response. getRetryInterval takes no arguments, and in particular has no
> access to the HttpResponse, so I’m not sure how I could go about using
> this. I had considered storing something when the call to retryRequest is
> made, but I’m not sure this is safe to do (it seems likely that it is not,
> since the strategy is almost certainly shared among multiple threads). It
> also seems kind of kludgy to me. Is there a way to do this cleanly? If not,
> is there a thread safe way to keep this state around between calls to the
> two methods?
>
>
--
BEKK Open
http://open.bekk.no
TesTcl - a unit test framework for iRules
http://testcl.com