You are viewing a plain text version of this content. The canonical link for it is here.
Posted to httpclient-users@hc.apache.org by "Brooks, Kenneth S" <ke...@chase.com> on 2010/05/11 20:53:18 UTC

stale connection

In the 4.0.1 tutorial I see the following paragraph.

... begin snippet ...
HttpClient tries to mitigate the problem by testing whether the connection is 'stale', that is no longer valid because it was closed on the server side, prior to using the connection for executing an HTTP request. The stale connection check is not 100% reliable and adds 10 to 30 ms overhead to each request execution. The only feasible solution that does not involve a one thread per socket model for idle connections is a dedicated monitor thread used to evict connections that are considered expired due to a long period of inactivity. The monitor thread can periodically call ClientConnectionManager#closeExpiredConnections() method to close all expired connections and evict closed connections from the pool. It can also optionally call ClientConnectionManager#closeIdleConnections() method to close all connections that have been idle over a given period of time.
... end snippet ...

Today we implement both the stale connection checking as well as the IdleConnectionTimeoutThread (in 3.1).
I would love to save the 10-30ms overhead but without doing that stale check it is possible that a connection was closed (that the Idle monitor may not have awoken to catch)..
Sounds like to me the only way to be as close to 100% safe is to implement both..

Is that correct?

-k




This transmission may contain information that is privileged,
confidential, legally privileged, and/or exempt from disclosure
under applicable law.  If you are not the intended recipient, you
are hereby notified that any disclosure, copying, distribution, or
use of the information contained herein (including any reliance
thereon) is STRICTLY PROHIBITED.  Although this transmission and
any attachments are believed to be free of any virus or other
defect that might affect any computer system into which it is
received and opened, it is the responsibility of the recipient to
ensure that it is virus free and no responsibility is accepted by
JPMorgan Chase & Co., its subsidiaries and affiliates, as
applicable, for any loss or damage arising in any way from its use.
 If you received this transmission in error, please immediately
contact the sender and destroy the material in its entirety,
whether in electronic or hard copy format. Thank you.

RE: stale connection

Posted by Oleg Kalnichevski <ol...@apache.org>.
On Wed, 2010-05-12 at 16:33 -0400, Brooks, Kenneth S wrote:
> Thanks for the responses. 
> 
> In the case of the StaleChecking, we also have an IdleConnectionMonitorThread that will evict connections that are over 20,000ms.
> We can do this in our environment with relative confidence because we control all the servers and those servers are set to close connections at 30,000ms.
> That should help to limit the number of 'Connection Resets' we have.
> 
> 
> So I guess it comes down to: if I wanted to remove my stalechecking and solely rely on retry handler, what exception would I expect to see if the connection was stale?
> That way I can account for that in the retry handler.
> 

Beat me with a stick, but I just cant say for sure. I think it can be
just a plain IOException, which basically means there is no way of
telling I/O errors caused by a stale connection from I/O errors caused
by connectivity issues.

Oleg  



---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org


RE: stale connection

Posted by "Brooks, Kenneth S" <ke...@chase.com>.
Thanks for the responses. 

In the case of the StaleChecking, we also have an IdleConnectionMonitorThread that will evict connections that are over 20,000ms.
We can do this in our environment with relative confidence because we control all the servers and those servers are set to close connections at 30,000ms.
That should help to limit the number of 'Connection Resets' we have.


So I guess it comes down to: if I wanted to remove my stalechecking and solely rely on retry handler, what exception would I expect to see if the connection was stale?
That way I can account for that in the retry handler.

-k


-----Original Message-----
From: Ken Krugler [mailto:kkrugler_lists@transpac.com] 
Sent: Wednesday, May 12, 2010 4:18 PM
To: HttpClient User Discussion
Subject: Re: stale connection


On May 12, 2010, at 12:28pm, Brooks, Kenneth S wrote:

> If the request made it to the server then we should not be sending  
> another request.
> 1) the server didn't actually process the message..
> 2) the server did process it but didn't return the response  
> correctly..
>
> In either of those cases, while they are frustrating.. we shouldn't  
> be resending.. because we have no mechanism on the server side to  
> identify duplicate messages.
> We might have the client write to an offline store that would  
> reconcile those failed messages later (which we have in some of our  
> applications).
>
>
> Ideally we would *only* retry a request under the following conditions
> 1) failure to make a connection to the server (connect timeout or a  
> stale connection)
> 2) when a request was not fully transmitted to the server
>
>
> So what combination would comply with those requirements?
> StaleChecking [yes/no]

Stale checking only helps reduce the probability of getting an  
IOException - the connection could still go stale in the window of  
time between the check and the request. So it's best to use a retry  
handler.

> DefaultRetryHandler [yes/no]

Unfortunately, as Oleg has tried to explain, for an exception like  
IOException isn't possible to know the difference between "it didn't  
make it to the server" and "the response from the server didn't make  
it back".

So while you can check NoHttpResponseException in the retry handler  
and retry those, you can't do it for the much more common IOException  
case.

-- Ken


> -----Original Message-----
> From: Oleg Kalnichevski [mailto:olegk@apache.org]
> Sent: Wednesday, May 12, 2010 2:47 PM
> To: HttpClient User Discussion
> Subject: RE: stale connection
>
> On Tue, 2010-05-11 at 22:04 -0400, Brooks, Kenneth S wrote:
>> We are doing serialization over http..
>> This means that 100% of our calls will *not* be idempotent..
>>
>> I don't see how we can avoid the stale check.
>> Are you saying that NoHttpResponseException is __always__ safe to  
>> retry?
>>
>
> Yes, it is
>
>> I can't take the risk of having a transaction submitted twice..
>
>
> You have that risk _anyways_ unless your application _never_  
> attempts to
> re-execute a failed request.
>
> HTTP is not a transactional transport. In complex network setups It  
> can
> also happen that the client gets an I/O error, even if the message has
> been successfully received and processed by the target server.
>
> You basically have two options: tolerate loss of messages or be  
> prepared
> to handle multiple message submissions.
>
> If your application cannot take action upon the same message twice,  
> the
> only possibility I personally can think of is the use of an unique
> identifier per request message.
>
> Oleg
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>
>
> This transmission may contain information that is privileged,
> confidential, legally privileged, and/or exempt from disclosure
> under applicable law.  If you are not the intended recipient, you
> are hereby notified that any disclosure, copying, distribution, or
> use of the information contained herein (including any reliance
> thereon) is STRICTLY PROHIBITED.  Although this transmission and
> any attachments are believed to be free of any virus or other
> defect that might affect any computer system into which it is
> received and opened, it is the responsibility of the recipient to
> ensure that it is virus free and no responsibility is accepted by
> JPMorgan Chase & Co., its subsidiaries and affiliates, as
> applicable, for any loss or damage arising in any way from its use.
> If you received this transmission in error, please immediately
> contact the sender and destroy the material in its entirety,
> whether in electronic or hard copy format. Thank you.

--------------------------------------------
Ken Krugler
+1 530-210-6378
http://bixolabs.com
e l a s t i c   w e b   m i n i n g





---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org


Re: stale connection

Posted by Ken Krugler <kk...@transpac.com>.
On May 12, 2010, at 12:28pm, Brooks, Kenneth S wrote:

> If the request made it to the server then we should not be sending  
> another request.
> 1) the server didn't actually process the message..
> 2) the server did process it but didn't return the response  
> correctly..
>
> In either of those cases, while they are frustrating.. we shouldn't  
> be resending.. because we have no mechanism on the server side to  
> identify duplicate messages.
> We might have the client write to an offline store that would  
> reconcile those failed messages later (which we have in some of our  
> applications).
>
>
> Ideally we would *only* retry a request under the following conditions
> 1) failure to make a connection to the server (connect timeout or a  
> stale connection)
> 2) when a request was not fully transmitted to the server
>
>
> So what combination would comply with those requirements?
> StaleChecking [yes/no]

Stale checking only helps reduce the probability of getting an  
IOException - the connection could still go stale in the window of  
time between the check and the request. So it's best to use a retry  
handler.

> DefaultRetryHandler [yes/no]

Unfortunately, as Oleg has tried to explain, for an exception like  
IOException isn't possible to know the difference between "it didn't  
make it to the server" and "the response from the server didn't make  
it back".

So while you can check NoHttpResponseException in the retry handler  
and retry those, you can't do it for the much more common IOException  
case.

-- Ken


> -----Original Message-----
> From: Oleg Kalnichevski [mailto:olegk@apache.org]
> Sent: Wednesday, May 12, 2010 2:47 PM
> To: HttpClient User Discussion
> Subject: RE: stale connection
>
> On Tue, 2010-05-11 at 22:04 -0400, Brooks, Kenneth S wrote:
>> We are doing serialization over http..
>> This means that 100% of our calls will *not* be idempotent..
>>
>> I don't see how we can avoid the stale check.
>> Are you saying that NoHttpResponseException is __always__ safe to  
>> retry?
>>
>
> Yes, it is
>
>> I can't take the risk of having a transaction submitted twice..
>
>
> You have that risk _anyways_ unless your application _never_  
> attempts to
> re-execute a failed request.
>
> HTTP is not a transactional transport. In complex network setups It  
> can
> also happen that the client gets an I/O error, even if the message has
> been successfully received and processed by the target server.
>
> You basically have two options: tolerate loss of messages or be  
> prepared
> to handle multiple message submissions.
>
> If your application cannot take action upon the same message twice,  
> the
> only possibility I personally can think of is the use of an unique
> identifier per request message.
>
> Oleg
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>
>
> This transmission may contain information that is privileged,
> confidential, legally privileged, and/or exempt from disclosure
> under applicable law.  If you are not the intended recipient, you
> are hereby notified that any disclosure, copying, distribution, or
> use of the information contained herein (including any reliance
> thereon) is STRICTLY PROHIBITED.  Although this transmission and
> any attachments are believed to be free of any virus or other
> defect that might affect any computer system into which it is
> received and opened, it is the responsibility of the recipient to
> ensure that it is virus free and no responsibility is accepted by
> JPMorgan Chase & Co., its subsidiaries and affiliates, as
> applicable, for any loss or damage arising in any way from its use.
> If you received this transmission in error, please immediately
> contact the sender and destroy the material in its entirety,
> whether in electronic or hard copy format. Thank you.

--------------------------------------------
Ken Krugler
+1 530-210-6378
http://bixolabs.com
e l a s t i c   w e b   m i n i n g





---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org


RE: stale connection

Posted by "Brooks, Kenneth S" <ke...@chase.com>.
If the request made it to the server then we should not be sending another request.
1) the server didn't actually process the message.. 
2) the server did process it but didn't return the response correctly..

In either of those cases, while they are frustrating.. we shouldn't be resending.. because we have no mechanism on the server side to identify duplicate messages.
We might have the client write to an offline store that would reconcile those failed messages later (which we have in some of our applications).


Ideally we would *only* retry a request under the following conditions
1) failure to make a connection to the server (connect timeout or a stale connection)
2) when a request was not fully transmitted to the server


So what combination would comply with those requirements?
StaleChecking [yes/no]
DefaultRetryHandler [yes/no]


-k

-----Original Message-----
From: Oleg Kalnichevski [mailto:olegk@apache.org] 
Sent: Wednesday, May 12, 2010 2:47 PM
To: HttpClient User Discussion
Subject: RE: stale connection

On Tue, 2010-05-11 at 22:04 -0400, Brooks, Kenneth S wrote:
> We are doing serialization over http..
> This means that 100% of our calls will *not* be idempotent.. 
> 
> I don't see how we can avoid the stale check. 
> Are you saying that NoHttpResponseException is __always__ safe to retry?
> 

Yes, it is

> I can't take the risk of having a transaction submitted twice.. 


You have that risk _anyways_ unless your application _never_ attempts to
re-execute a failed request.

HTTP is not a transactional transport. In complex network setups It can
also happen that the client gets an I/O error, even if the message has
been successfully received and processed by the target server.  

You basically have two options: tolerate loss of messages or be prepared
to handle multiple message submissions. 

If your application cannot take action upon the same message twice, the
only possibility I personally can think of is the use of an unique
identifier per request message. 

Oleg


---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org


This transmission may contain information that is privileged,
confidential, legally privileged, and/or exempt from disclosure
under applicable law.  If you are not the intended recipient, you
are hereby notified that any disclosure, copying, distribution, or
use of the information contained herein (including any reliance
thereon) is STRICTLY PROHIBITED.  Although this transmission and
any attachments are believed to be free of any virus or other
defect that might affect any computer system into which it is
received and opened, it is the responsibility of the recipient to
ensure that it is virus free and no responsibility is accepted by
JPMorgan Chase & Co., its subsidiaries and affiliates, as
applicable, for any loss or damage arising in any way from its use.
 If you received this transmission in error, please immediately
contact the sender and destroy the material in its entirety,
whether in electronic or hard copy format. Thank you.

RE: stale connection

Posted by Oleg Kalnichevski <ol...@apache.org>.
On Tue, 2010-05-11 at 22:04 -0400, Brooks, Kenneth S wrote:
> We are doing serialization over http..
> This means that 100% of our calls will *not* be idempotent.. 
> 
> I don't see how we can avoid the stale check. 
> Are you saying that NoHttpResponseException is __always__ safe to retry?
> 

Yes, it is

> I can't take the risk of having a transaction submitted twice.. 


You have that risk _anyways_ unless your application _never_ attempts to
re-execute a failed request.

HTTP is not a transactional transport. In complex network setups It can
also happen that the client gets an I/O error, even if the message has
been successfully received and processed by the target server.  

You basically have two options: tolerate loss of messages or be prepared
to handle multiple message submissions. 

If your application cannot take action upon the same message twice, the
only possibility I personally can think of is the use of an unique
identifier per request message. 

Oleg


---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org


RE: stale connection

Posted by "Brooks, Kenneth S" <ke...@chase.com>.
We are doing serialization over http..
This means that 100% of our calls will *not* be idempotent.. 

I don't see how we can avoid the stale check. 
Are you saying that NoHttpResponseException is __always__ safe to retry?

I can't take the risk of having a transaction submitted twice.. 

-k

-----Original Message-----
From: Oleg Kalnichevski [mailto:olegk@apache.org] 
Sent: Tuesday, May 11, 2010 4:50 PM
To: HttpClient User Discussion
Subject: Re: stale connection

On Tue, 2010-05-11 at 12:28 -0700, Ken Krugler wrote:
> On May 11, 2010, at 11:53am, Brooks, Kenneth S wrote:
> 
> > In the 4.0.1 tutorial I see the following paragraph.
> >
> > ... begin snippet ...
> > HttpClient tries to mitigate the problem by testing whether the  
> > connection is 'stale', that is no longer valid because it was closed  
> > on the server side, prior to using the connection for executing an  
> > HTTP request. The stale connection check is not 100% reliable and  
> > adds 10 to 30 ms overhead to each request execution. The only  
> > feasible solution that does not involve a one thread per socket  
> > model for idle connections is a dedicated monitor thread used to  
> > evict connections that are considered expired due to a long period  
> > of inactivity. The monitor thread can periodically call  
> > ClientConnectionManager#closeExpiredConnections() method to close  
> > all expired connections and evict closed connections from the pool.  
> > It can also optionally call  
> > ClientConnectionManager#closeIdleConnections() method to close all  
> > connections that have been idle over a given period of time.
> > ... end snippet ...
> >
> > Today we implement both the stale connection checking as well as the  
> > IdleConnectionTimeoutThread (in 3.1).
> > I would love to save the 10-30ms overhead but without doing that  
> > stale check it is possible that a connection was closed (that the  
> > Idle monitor may not have awoken to catch)..
> > Sounds like to me the only way to be as close to 100% safe is to  
> > implement both..
> >
> > Is that correct?
> 
> I don't believe so.
> 
> Assuming the connection is stale, then you'll get an exception when  
> you try to use it, so assuming you have a retry handler installed, you  
> can automatically retry that type of failure.
> 
> Some code from the Bixo project to illustrate:
> 
>              _httpClient = new DefaultHttpClient(cm, params);
>              _httpClient.setHttpRequestRetryHandler(new  
> MyRequestRetryHandler(MAX_RETRY_COUNT));
> 
> 
>      private static class MyRequestRetryHandler implements  
> HttpRequestRetryHandler {
>          private int _maxRetryCount;
> 
>          public MyRequestRetryHandler(int maxRetryCount) {
>              _maxRetryCount = maxRetryCount;
>          }
> 
>          @Override
>          public boolean retryRequest(IOException exception, int  
> executionCount, HttpContext context) {
>              if (LOGGER.isTraceEnabled()) {
>                  LOGGER.trace("Decide about retry #" + executionCount  
> + " for exception " + exception.getMessage());
>              }
> 
>              if (executionCount >= _maxRetryCount) {
>                  // Do not retry if over max retry count
>                  return false;
>              } else if (exception instanceof NoHttpResponseException) {
>                  // Retry if the server dropped connection on us
>                  return true;
>              } else if (exception instanceof SSLHandshakeException) {
>                  // Do not retry on SSL handshake exception
>                  return false;
>              }
> 
>              HttpRequest request =  
> (HttpRequest)context.getAttribute(ExecutionContext.HTTP_REQUEST);
>              boolean idempotent = !(request instanceof  
> HttpEntityEnclosingRequest);
>              // Retry if the request is considered idempotent
>              return idempotent;
>          }
>      }
> 

Yep, a retry handler is the way to go. 

Oleg

> 
> --------------------------------------------
> Ken Krugler
> +1 530-210-6378
> http://bixolabs.com
> e l a s t i c   w e b   m i n i n g
> 
> 
> 
> 



---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org


This transmission may contain information that is privileged,
confidential, legally privileged, and/or exempt from disclosure
under applicable law.  If you are not the intended recipient, you
are hereby notified that any disclosure, copying, distribution, or
use of the information contained herein (including any reliance
thereon) is STRICTLY PROHIBITED.  Although this transmission and
any attachments are believed to be free of any virus or other
defect that might affect any computer system into which it is
received and opened, it is the responsibility of the recipient to
ensure that it is virus free and no responsibility is accepted by
JPMorgan Chase & Co., its subsidiaries and affiliates, as
applicable, for any loss or damage arising in any way from its use.
 If you received this transmission in error, please immediately
contact the sender and destroy the material in its entirety,
whether in electronic or hard copy format. Thank you.

Re: stale connection

Posted by Oleg Kalnichevski <ol...@apache.org>.
On Tue, 2010-05-11 at 12:28 -0700, Ken Krugler wrote:
> On May 11, 2010, at 11:53am, Brooks, Kenneth S wrote:
> 
> > In the 4.0.1 tutorial I see the following paragraph.
> >
> > ... begin snippet ...
> > HttpClient tries to mitigate the problem by testing whether the  
> > connection is 'stale', that is no longer valid because it was closed  
> > on the server side, prior to using the connection for executing an  
> > HTTP request. The stale connection check is not 100% reliable and  
> > adds 10 to 30 ms overhead to each request execution. The only  
> > feasible solution that does not involve a one thread per socket  
> > model for idle connections is a dedicated monitor thread used to  
> > evict connections that are considered expired due to a long period  
> > of inactivity. The monitor thread can periodically call  
> > ClientConnectionManager#closeExpiredConnections() method to close  
> > all expired connections and evict closed connections from the pool.  
> > It can also optionally call  
> > ClientConnectionManager#closeIdleConnections() method to close all  
> > connections that have been idle over a given period of time.
> > ... end snippet ...
> >
> > Today we implement both the stale connection checking as well as the  
> > IdleConnectionTimeoutThread (in 3.1).
> > I would love to save the 10-30ms overhead but without doing that  
> > stale check it is possible that a connection was closed (that the  
> > Idle monitor may not have awoken to catch)..
> > Sounds like to me the only way to be as close to 100% safe is to  
> > implement both..
> >
> > Is that correct?
> 
> I don't believe so.
> 
> Assuming the connection is stale, then you'll get an exception when  
> you try to use it, so assuming you have a retry handler installed, you  
> can automatically retry that type of failure.
> 
> Some code from the Bixo project to illustrate:
> 
>              _httpClient = new DefaultHttpClient(cm, params);
>              _httpClient.setHttpRequestRetryHandler(new  
> MyRequestRetryHandler(MAX_RETRY_COUNT));
> 
> 
>      private static class MyRequestRetryHandler implements  
> HttpRequestRetryHandler {
>          private int _maxRetryCount;
> 
>          public MyRequestRetryHandler(int maxRetryCount) {
>              _maxRetryCount = maxRetryCount;
>          }
> 
>          @Override
>          public boolean retryRequest(IOException exception, int  
> executionCount, HttpContext context) {
>              if (LOGGER.isTraceEnabled()) {
>                  LOGGER.trace("Decide about retry #" + executionCount  
> + " for exception " + exception.getMessage());
>              }
> 
>              if (executionCount >= _maxRetryCount) {
>                  // Do not retry if over max retry count
>                  return false;
>              } else if (exception instanceof NoHttpResponseException) {
>                  // Retry if the server dropped connection on us
>                  return true;
>              } else if (exception instanceof SSLHandshakeException) {
>                  // Do not retry on SSL handshake exception
>                  return false;
>              }
> 
>              HttpRequest request =  
> (HttpRequest)context.getAttribute(ExecutionContext.HTTP_REQUEST);
>              boolean idempotent = !(request instanceof  
> HttpEntityEnclosingRequest);
>              // Retry if the request is considered idempotent
>              return idempotent;
>          }
>      }
> 

Yep, a retry handler is the way to go. 

Oleg

> 
> --------------------------------------------
> Ken Krugler
> +1 530-210-6378
> http://bixolabs.com
> e l a s t i c   w e b   m i n i n g
> 
> 
> 
> 



---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org


Re: stale connection

Posted by Ken Krugler <kk...@transpac.com>.
On May 11, 2010, at 11:53am, Brooks, Kenneth S wrote:

> In the 4.0.1 tutorial I see the following paragraph.
>
> ... begin snippet ...
> HttpClient tries to mitigate the problem by testing whether the  
> connection is 'stale', that is no longer valid because it was closed  
> on the server side, prior to using the connection for executing an  
> HTTP request. The stale connection check is not 100% reliable and  
> adds 10 to 30 ms overhead to each request execution. The only  
> feasible solution that does not involve a one thread per socket  
> model for idle connections is a dedicated monitor thread used to  
> evict connections that are considered expired due to a long period  
> of inactivity. The monitor thread can periodically call  
> ClientConnectionManager#closeExpiredConnections() method to close  
> all expired connections and evict closed connections from the pool.  
> It can also optionally call  
> ClientConnectionManager#closeIdleConnections() method to close all  
> connections that have been idle over a given period of time.
> ... end snippet ...
>
> Today we implement both the stale connection checking as well as the  
> IdleConnectionTimeoutThread (in 3.1).
> I would love to save the 10-30ms overhead but without doing that  
> stale check it is possible that a connection was closed (that the  
> Idle monitor may not have awoken to catch)..
> Sounds like to me the only way to be as close to 100% safe is to  
> implement both..
>
> Is that correct?

I don't believe so.

Assuming the connection is stale, then you'll get an exception when  
you try to use it, so assuming you have a retry handler installed, you  
can automatically retry that type of failure.

Some code from the Bixo project to illustrate:

             _httpClient = new DefaultHttpClient(cm, params);
             _httpClient.setHttpRequestRetryHandler(new  
MyRequestRetryHandler(MAX_RETRY_COUNT));


     private static class MyRequestRetryHandler implements  
HttpRequestRetryHandler {
         private int _maxRetryCount;

         public MyRequestRetryHandler(int maxRetryCount) {
             _maxRetryCount = maxRetryCount;
         }

         @Override
         public boolean retryRequest(IOException exception, int  
executionCount, HttpContext context) {
             if (LOGGER.isTraceEnabled()) {
                 LOGGER.trace("Decide about retry #" + executionCount  
+ " for exception " + exception.getMessage());
             }

             if (executionCount >= _maxRetryCount) {
                 // Do not retry if over max retry count
                 return false;
             } else if (exception instanceof NoHttpResponseException) {
                 // Retry if the server dropped connection on us
                 return true;
             } else if (exception instanceof SSLHandshakeException) {
                 // Do not retry on SSL handshake exception
                 return false;
             }

             HttpRequest request =  
(HttpRequest)context.getAttribute(ExecutionContext.HTTP_REQUEST);
             boolean idempotent = !(request instanceof  
HttpEntityEnclosingRequest);
             // Retry if the request is considered idempotent
             return idempotent;
         }
     }


--------------------------------------------
Ken Krugler
+1 530-210-6378
http://bixolabs.com
e l a s t i c   w e b   m i n i n g