You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@jclouds.apache.org by Daniel Hsueh <Da...@evault.com> on 2014/09/10 15:38:14 UTC

use exponential backoff on retry (was RE: JClouds BlobStore re-authentication when auth token expires)

Hello everyone,

Is there a way to configure the lengths of the delay in between retry attempts on auth, read, or write operations?

Thank you.


Daniel Hsueh
mailto:daniel.hsueh@evault.com tel:+1-905-287-2167

-----Original Message-----
From: Daniel Hsueh 
Sent: September 2, 2014 11:38 AM
To: 'user@jclouds.apache.org'
Subject: RE: JClouds BlobStore re-authentication when auth token expires

[snip]

One thing I'd like to configure is some kind of exponential backoff when the auth retry fails.  IIRC, there are three RetryOnRenew classes, and they either retry 5 times, or retry 5 times with a fixed delay between the 2,3,4th attempts.  Is there a way to configure an increasing backoff?

Thanks!


Daniel Hsueh
mailto:daniel.hsueh@evault.com tel:+1-905-287-2167

Re: use exponential backoff on retry (was RE: JClouds BlobStore re-authentication when auth token expires)

Posted by Yury Kats <yu...@yahoo.com>.
CONNECTION_TIMEOUT applies to establishing a connection
SO_TIMEOUT applies to existing connection that is idle.

On 10/17/2014 12:32 PM, Zack Shoylev wrote:
> They are slightly different. I think the connection timeout only applies when initiating the connection to the remote host. I don't think they are additive.
> ________________________________________
> From: Daniel Hsueh [Daniel.Hsueh@evault.com]
> Sent: Wednesday, October 15, 2014 11:02 AM
> To: user@jclouds.apache.org
> Subject: RE: use exponential backoff on retry (was RE: JClouds BlobStore re-authentication when auth token expires)
> 
> Hello Zack,
> 
> Thank you for the explanation.
> 
> Could you clarify if, at each retry (after the delay), the SO_TIMEOUT (or CONNECTION_TIMEOUT) is waited-for as well?  And what is the difference between the SO_TIMEOUT and the CONNECTION_TIMEOUT?  (Comments in Constants.java say socket connection vs connection, code [e.g. HttpUtils] says "so" is socket open, and I can't seem to trace exactly where the values end up being used in practice.)
> 
> Thank you again.
> 
> Daniel Hsueh
> mailto:daniel.hsueh@evault.com tel:+1-905-287-2167
> 
> -----Original Message-----
> From: Zack Shoylev [mailto:zack.shoylev@RACKSPACE.COM]
> Sent: September 10, 2014 11:48 AM
> To: user@jclouds.apache.org
> Subject: RE: use exponential backoff on retry (was RE: JClouds BlobStore re-authentication when auth token expires)
> 
> Hi Daniel,
> 
> You should try testing with these Properties:
> 
> PROPERTY_SO_TIMEOUT - This will time-out the HTTP connection socket after the specified time. This specifies how long the client should wait before jclouds closes the connection and attempts to retry the HTTP call. I would suggest a value of 5 seconds.
> 
> PROPERTY_MAX_RETRIES - This specifies the maximum number of retries for the call that will be attempted by jclouds. Note that setting this to 0 will result in a single call. Setting this to 1 will retry once before failing.
> 
> PROPERTY_RETRY_DELAY_START - This specifies how long the exponential backoff waits between retries. The exponential backoff retry mechanism in jclouds will double this value after each retry.
> 
> I don't think you can set them differently for different operations, though.
> 
> -Zack
> ________________________________________
> From: Daniel Hsueh [Daniel.Hsueh@evault.com]
> Sent: Wednesday, September 10, 2014 8:38 AM
> To: user@jclouds.apache.org
> Subject: use exponential backoff on retry (was RE: JClouds BlobStore re-authentication when auth token expires)
> 
> Hello everyone,
> 
> Is there a way to configure the lengths of the delay in between retry attempts on auth, read, or write operations?
> 
> Thank you.
> 
> 
> Daniel Hsueh
> mailto:daniel.hsueh@evault.com tel:+1-905-287-2167
> 
> -----Original Message-----
> From: Daniel Hsueh
> Sent: September 2, 2014 11:38 AM
> To: 'user@jclouds.apache.org'
> Subject: RE: JClouds BlobStore re-authentication when auth token expires
> 
> [snip]
> 
> One thing I'd like to configure is some kind of exponential backoff when the auth retry fails.  IIRC, there are three RetryOnRenew classes, and they either retry 5 times, or retry 5 times with a fixed delay between the 2,3,4th attempts.  Is there a way to configure an increasing backoff?
> 
> Thanks!
> 
> 
> Daniel Hsueh
> mailto:daniel.hsueh@evault.com tel:+1-905-287-2167
> 
>

RE: use exponential backoff on retry (was RE: JClouds BlobStore re-authentication when auth token expires)

Posted by Zack Shoylev <za...@RACKSPACE.COM>.
They are slightly different. I think the connection timeout only applies when initiating the connection to the remote host. I don't think they are additive.
________________________________________
From: Daniel Hsueh [Daniel.Hsueh@evault.com]
Sent: Wednesday, October 15, 2014 11:02 AM
To: user@jclouds.apache.org
Subject: RE: use exponential backoff on retry (was RE: JClouds BlobStore re-authentication when auth token expires)

Hello Zack,

Thank you for the explanation.

Could you clarify if, at each retry (after the delay), the SO_TIMEOUT (or CONNECTION_TIMEOUT) is waited-for as well?  And what is the difference between the SO_TIMEOUT and the CONNECTION_TIMEOUT?  (Comments in Constants.java say socket connection vs connection, code [e.g. HttpUtils] says "so" is socket open, and I can't seem to trace exactly where the values end up being used in practice.)

Thank you again.

Daniel Hsueh
mailto:daniel.hsueh@evault.com tel:+1-905-287-2167

-----Original Message-----
From: Zack Shoylev [mailto:zack.shoylev@RACKSPACE.COM]
Sent: September 10, 2014 11:48 AM
To: user@jclouds.apache.org
Subject: RE: use exponential backoff on retry (was RE: JClouds BlobStore re-authentication when auth token expires)

Hi Daniel,

You should try testing with these Properties:

PROPERTY_SO_TIMEOUT - This will time-out the HTTP connection socket after the specified time. This specifies how long the client should wait before jclouds closes the connection and attempts to retry the HTTP call. I would suggest a value of 5 seconds.

PROPERTY_MAX_RETRIES - This specifies the maximum number of retries for the call that will be attempted by jclouds. Note that setting this to 0 will result in a single call. Setting this to 1 will retry once before failing.

PROPERTY_RETRY_DELAY_START - This specifies how long the exponential backoff waits between retries. The exponential backoff retry mechanism in jclouds will double this value after each retry.

I don't think you can set them differently for different operations, though.

-Zack
________________________________________
From: Daniel Hsueh [Daniel.Hsueh@evault.com]
Sent: Wednesday, September 10, 2014 8:38 AM
To: user@jclouds.apache.org
Subject: use exponential backoff on retry (was RE: JClouds BlobStore re-authentication when auth token expires)

Hello everyone,

Is there a way to configure the lengths of the delay in between retry attempts on auth, read, or write operations?

Thank you.


Daniel Hsueh
mailto:daniel.hsueh@evault.com tel:+1-905-287-2167

-----Original Message-----
From: Daniel Hsueh
Sent: September 2, 2014 11:38 AM
To: 'user@jclouds.apache.org'
Subject: RE: JClouds BlobStore re-authentication when auth token expires

[snip]

One thing I'd like to configure is some kind of exponential backoff when the auth retry fails.  IIRC, there are three RetryOnRenew classes, and they either retry 5 times, or retry 5 times with a fixed delay between the 2,3,4th attempts.  Is there a way to configure an increasing backoff?

Thanks!


Daniel Hsueh
mailto:daniel.hsueh@evault.com tel:+1-905-287-2167

RE: use exponential backoff on retry (was RE: JClouds BlobStore re-authentication when auth token expires)

Posted by Daniel Hsueh <Da...@evault.com>.
Hello Zack,

Thank you for the explanation.

Could you clarify if, at each retry (after the delay), the SO_TIMEOUT (or CONNECTION_TIMEOUT) is waited-for as well?  And what is the difference between the SO_TIMEOUT and the CONNECTION_TIMEOUT?  (Comments in Constants.java say socket connection vs connection, code [e.g. HttpUtils] says "so" is socket open, and I can't seem to trace exactly where the values end up being used in practice.)

Thank you again.

Daniel Hsueh
mailto:daniel.hsueh@evault.com tel:+1-905-287-2167

-----Original Message-----
From: Zack Shoylev [mailto:zack.shoylev@RACKSPACE.COM] 
Sent: September 10, 2014 11:48 AM
To: user@jclouds.apache.org
Subject: RE: use exponential backoff on retry (was RE: JClouds BlobStore re-authentication when auth token expires)

Hi Daniel,

You should try testing with these Properties:

PROPERTY_SO_TIMEOUT - This will time-out the HTTP connection socket after the specified time. This specifies how long the client should wait before jclouds closes the connection and attempts to retry the HTTP call. I would suggest a value of 5 seconds.

PROPERTY_MAX_RETRIES - This specifies the maximum number of retries for the call that will be attempted by jclouds. Note that setting this to 0 will result in a single call. Setting this to 1 will retry once before failing. 

PROPERTY_RETRY_DELAY_START - This specifies how long the exponential backoff waits between retries. The exponential backoff retry mechanism in jclouds will double this value after each retry.

I don't think you can set them differently for different operations, though.

-Zack
________________________________________
From: Daniel Hsueh [Daniel.Hsueh@evault.com]
Sent: Wednesday, September 10, 2014 8:38 AM
To: user@jclouds.apache.org
Subject: use exponential backoff on retry (was RE: JClouds BlobStore re-authentication when auth token expires)

Hello everyone,

Is there a way to configure the lengths of the delay in between retry attempts on auth, read, or write operations?

Thank you.


Daniel Hsueh
mailto:daniel.hsueh@evault.com tel:+1-905-287-2167

-----Original Message-----
From: Daniel Hsueh
Sent: September 2, 2014 11:38 AM
To: 'user@jclouds.apache.org'
Subject: RE: JClouds BlobStore re-authentication when auth token expires

[snip]

One thing I'd like to configure is some kind of exponential backoff when the auth retry fails.  IIRC, there are three RetryOnRenew classes, and they either retry 5 times, or retry 5 times with a fixed delay between the 2,3,4th attempts.  Is there a way to configure an increasing backoff?

Thanks!


Daniel Hsueh
mailto:daniel.hsueh@evault.com tel:+1-905-287-2167

RE: use exponential backoff on retry (was RE: JClouds BlobStore re-authentication when auth token expires)

Posted by Zack Shoylev <za...@RACKSPACE.COM>.
Hi Daniel,

You should try testing with these Properties:

PROPERTY_SO_TIMEOUT - This will time-out the HTTP connection socket after the specified time. This specifies how long the client should wait before jclouds closes the connection and attempts to retry the HTTP call. I would suggest a value of 5 seconds.

PROPERTY_MAX_RETRIES - This specifies the maximum number of retries for the call that will be attempted by jclouds. Note that setting this to 0 will result in a single call. Setting this to 1 will retry once before failing. 

PROPERTY_RETRY_DELAY_START - This specifies how long the exponential backoff waits between retries. The exponential backoff retry mechanism in jclouds will double this value after each retry.

I don't think you can set them differently for different operations, though.

-Zack
________________________________________
From: Daniel Hsueh [Daniel.Hsueh@evault.com]
Sent: Wednesday, September 10, 2014 8:38 AM
To: user@jclouds.apache.org
Subject: use exponential backoff on retry (was RE: JClouds BlobStore re-authentication when auth token expires)

Hello everyone,

Is there a way to configure the lengths of the delay in between retry attempts on auth, read, or write operations?

Thank you.


Daniel Hsueh
mailto:daniel.hsueh@evault.com tel:+1-905-287-2167

-----Original Message-----
From: Daniel Hsueh
Sent: September 2, 2014 11:38 AM
To: 'user@jclouds.apache.org'
Subject: RE: JClouds BlobStore re-authentication when auth token expires

[snip]

One thing I'd like to configure is some kind of exponential backoff when the auth retry fails.  IIRC, there are three RetryOnRenew classes, and they either retry 5 times, or retry 5 times with a fixed delay between the 2,3,4th attempts.  Is there a way to configure an increasing backoff?

Thanks!


Daniel Hsueh
mailto:daniel.hsueh@evault.com tel:+1-905-287-2167