You are viewing a plain text version of this content. The canonical link for it is here.
Posted to httpclient-users@hc.apache.org by Daniel Lapadat <Da...@baesystems.com> on 2006/11/08 20:13:28 UTC

httpclient proxy timeouts with long responses

I have not been able to find a solution to the following problem.

We have an application that talks to servlets using HTTPClient.  It requests
information from the servlets, sometimes the response can take a long time ( >
10 minutes ).  Everything was working fine until some of our new users sat
behind a proxy server.  The proxy server is timing out our HTTP connection at
around 5 minutes.  We of course cannot modify any of the proxy server's
settings.  Is there some way of sending a keep alive signal to the proxy server?
 I thought that HTTP 1.1 automatically did this.

The only solution we have found is to start writing white space back to the
client every couple of seconds but if we do that and there is an error on the
server side, the client will not get any response back from the server because
we started to write to the output stream of the HTTPResponse object.  sendError
causes an IllegalStateException on the server end.  In order to avoid this we
cannot write to the output stream until we have a response from the database or
other systems.

Any help would be appreciated,

Dan

---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-user-help@jakarta.apache.org


Re: httpclient proxy timeouts with long responses

Posted by Roland Weber <ht...@dubioso.net>.
Hello Daniel,

I seem to be a little late for this discussion, but here
are my 0.02€ anyway.

If it wasn't the servlet API, you could try to send a
status 100 response every few minutes. I'm not sure what
the servlet API will do if you try to send intermediate
responses, but I guess it will fail one way or another.

The other idea has already been mentioned: process the
request asynchronously. If you get the result in time,
send it back directly. If you don't have the result after
n minutes, send a redirect that will be automatically
followed by a browser or HttpClient. The servlet to which
the redirect points then picks up the asynchronous request
that is being processed. If the result is not available
after another n minutes, repeat. As soon as you have the
result or a failure, generate the final response.

cheers,
  Roland



---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-user-help@jakarta.apache.org


AW: AW: httpclient proxy timeouts with long responses

Posted by Konstantin Breu <Ko...@gmx.net>.
> We are already using the first idea but have a problem if an error
> happens.  We cannot notify the client that the error has happened
> because we have already started to write to the stream.  The client now
> does not know what happened and will process everything like normal, in
> which case they get an empty string as the response, which results in a
> failure (expecting XML).
Well, if an error occurs you could also send error information in a
multipart response part - also as XML. But: I don't know in how far
HttpClient offers support for multipart responses. Hopefully the httpclient
gurus can answer that question. ;-) 
Anyway, I would prefer the way polling the server (which was my idea number
2). That's what I use in such situations in normal cases, and I think many
other developers, too. Whereas I don't know much real-world solutions using
http push.

Here is some info about pull + push:
http://wp.netscape.com/assist/net_sites/pushpull.html
http://www.unix.org.ua/orelly/web/html/ch14_03.html


> -----Ursprüngliche Nachricht-----
> Von: Daniel Lapadat [mailto:Daniel.Lapadat@baesystems.com]
> Gesendet: Mittwoch, 8. November 2006 23:02
> An: HttpClient User Discussion
> Betreff: Re: AW: httpclient proxy timeouts with long responses
> 
> Breu,
> 
> Thanks for the ideas.
> 
> We are already using the first idea but have a problem if an error
> happens.  We cannot notify the client that the error has happened
> because we have already started to write to the stream.  The client now
> does not know what happened and will process everything like normal, in
> which case they get an empty string as the response, which results in a
> failure (expecting XML).  On the server end we will get an
> IllegalStateException because we have already committed.  Also I
> believe that to use a multipart response I would need to have all of
> the data upfront, at least that is what it looks like in the API
> documentation for HTTPClient.
> 
> If I understand you, you are suggesting to take an asynchronous
> approach, which I would like to avoid as it would involve a rewrite.
> You are suggesting that we basically poll the server for when our
> information is done, right?
> 
> Dan
> 
> Konstantin Breu wrote:
> > 2 ideas:
> >
> > - kind of http push: send a multipart response; the first parts are
> > sent every x second/minute just to make the proxy keep the
> connection;
> > the last part contains the "real response"; of course the client will
> > have to look for the parts to get information about processing/result
> >
> > - or a solution with multiple requests during processing (every x
> > second/minute) until the response is generated
> >
> >> -----Ursprüngliche Nachricht-----
> >> Von: Daniel Lapadat [mailto:Daniel.Lapadat@baesystems.com]
> >> Gesendet: Mittwoch, 8. November 2006 20:13
> >> An: httpclient-user@jakarta.apache.org
> >> Betreff: httpclient proxy timeouts with long responses
> >>
> >> I have not been able to find a solution to the following problem.
> >>
> >> We have an application that talks to servlets using HTTPClient.  It
> >> requests information from the servlets, sometimes the response can
> >> take a long time ( > 10 minutes ).  Everything was working fine
> until
> >> some of our new users sat behind a proxy server.  The proxy server
> is
> >> timing out our HTTP connection at around 5 minutes.  We of course
> >> cannot modify any of the proxy server's settings.  Is there some way
> >> of sending a keep alive signal to the proxy server?
> >>  I thought that HTTP 1.1 automatically did this.
> >>
> >> The only solution we have found is to start writing white space back
> >> to the client every couple of seconds but if we do that and there is
> >> an error on the server side, the client will not get any response
> >> back from the server because we started to write to the output
> stream
> >> of the HTTPResponse object.  sendError causes an
> >> IllegalStateException on the server end.  In order to avoid this we
> >> cannot write to the output stream until we have a response from the
> database or other systems.
> >>
> >> Any help would be appreciated,
> >>
> >> Dan
> >>
> >> --------------------------------------------------------------------
> -
> >> To unsubscribe, e-mail:
> >> httpclient-user-unsubscribe@jakarta.apache.org
> >> For additional commands, e-mail: httpclient-user-
> >> help@jakarta.apache.org
> >
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: httpclient-user-
> unsubscribe@jakarta.apache.org
> > For additional commands, e-mail:
> > httpclient-user-help@jakarta.apache.org
> >
> 
> --
> Daniel Lapadat
> Software Engineer
> BAE Systems -- National Security Solutions 6310 Hillside Court Suite
> 200 Columbia, MD 21046
> Office: 410-309-9366 x 2096
> Fax:    410-309-9370
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: httpclient-user-
> help@jakarta.apache.org




---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-user-help@jakarta.apache.org


Re: AW: httpclient proxy timeouts with long responses

Posted by Daniel Lapadat <Da...@baesystems.com>.
I am referring to a rewriting parts of our application.  It would require
polling the server as well as creating a mechanism for handling server side errors.

I wanted to avoid this situation because it would involve a rewrite. Also
searches that are quick have to use this mechanism because we have no way of
predicting how long the search or response will take.  That is going to slow
them down.  A suitable poll interval must also be found.  Thanks for you help,

Dan

Oleg Kalnichevski wrote:
> On Wed, 2006-11-08 at 17:01 -0500, Daniel Lapadat wrote:
>> Breu,
>>
>> Thanks for the ideas.
>>
>> We are already using the first idea but have a problem if an error happens.  We
>> cannot notify the client that the error has happened because we have already
>> started to write to the stream.  The client now does not know what happened and
>> will process everything like normal, in which case they get an empty string as
>> the response, which results in a failure (expecting XML).  On the server end we
>> will get an IllegalStateException because we have already committed.  Also I
>> believe that to use a multipart response I would need to have all of the data
>> upfront, at least that is what it looks like in the API documentation for
>> HTTPClient.
>>
>> If I understand you, you are suggesting to take an asynchronous approach, which
>> I would like to avoid as it would involve a rewrite.  You are suggesting that we
>> basically poll the server for when our information is done, right?
>>
>> Dan
>>
> 
> Asynchronous processing of such requests would probably be the cleanest
> solution to the problem that does not involve any ugly hacks on the HTTP
> transport level. What kind of rewrite would that require? 
> 
> Oleg
> 
> 
>> Konstantin Breu wrote:
>>> 2 ideas:
>>>
>>> - kind of http push: send a multipart response; the first parts are sent
>>> every x second/minute just to make the proxy keep the connection; the last
>>> part contains the "real response"; of course the client will have to look
>>> for the parts to get information about processing/result
>>>
>>> - or a solution with multiple requests during processing (every x
>>> second/minute) until the response is generated
>>>
>>>> -----Ursprüngliche Nachricht-----
>>>> Von: Daniel Lapadat [mailto:Daniel.Lapadat@baesystems.com]
>>>> Gesendet: Mittwoch, 8. November 2006 20:13
>>>> An: httpclient-user@jakarta.apache.org
>>>> Betreff: httpclient proxy timeouts with long responses
>>>>
>>>> I have not been able to find a solution to the following problem.
>>>>
>>>> We have an application that talks to servlets using HTTPClient.  It
>>>> requests information from the servlets, sometimes the response can take
>>>> a long time ( > 10 minutes ).  Everything was working fine until some
>>>> of our new users sat behind a proxy server.  The proxy server is timing
>>>> out our HTTP connection at around 5 minutes.  We of course cannot
>>>> modify any of the proxy server's settings.  Is there some way of
>>>> sending a keep alive signal to the proxy server?
>>>>  I thought that HTTP 1.1 automatically did this.
>>>>
>>>> The only solution we have found is to start writing white space back to
>>>> the client every couple of seconds but if we do that and there is an
>>>> error on the server side, the client will not get any response back
>>>> from the server because we started to write to the output stream of the
>>>> HTTPResponse object.  sendError causes an IllegalStateException on the
>>>> server end.  In order to avoid this we cannot write to the output
>>>> stream until we have a response from the database or other systems.
>>>>
>>>> Any help would be appreciated,
>>>>
>>>> Dan
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
>>>> For additional commands, e-mail: httpclient-user-
>>>> help@jakarta.apache.org
>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
>>> For additional commands, e-mail: httpclient-user-help@jakarta.apache.org
>>>
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: httpclient-user-help@jakarta.apache.org
> 

-- 
Daniel Lapadat
Software Engineer
BAE Systems -- National Security Solutions
6310 Hillside Court
Suite 200
Columbia, MD 21046
Office: 410-309-9366 x 2096
Fax:    410-309-9370

---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-user-help@jakarta.apache.org


Re: AW: httpclient proxy timeouts with long responses

Posted by Oleg Kalnichevski <ol...@apache.org>.
On Wed, 2006-11-08 at 17:01 -0500, Daniel Lapadat wrote:
> Breu,
> 
> Thanks for the ideas.
> 
> We are already using the first idea but have a problem if an error happens.  We
> cannot notify the client that the error has happened because we have already
> started to write to the stream.  The client now does not know what happened and
> will process everything like normal, in which case they get an empty string as
> the response, which results in a failure (expecting XML).  On the server end we
> will get an IllegalStateException because we have already committed.  Also I
> believe that to use a multipart response I would need to have all of the data
> upfront, at least that is what it looks like in the API documentation for
> HTTPClient.
> 
> If I understand you, you are suggesting to take an asynchronous approach, which
> I would like to avoid as it would involve a rewrite.  You are suggesting that we
> basically poll the server for when our information is done, right?
> 
> Dan
> 

Asynchronous processing of such requests would probably be the cleanest
solution to the problem that does not involve any ugly hacks on the HTTP
transport level. What kind of rewrite would that require? 

Oleg


> Konstantin Breu wrote:
> > 2 ideas:
> > 
> > - kind of http push: send a multipart response; the first parts are sent
> > every x second/minute just to make the proxy keep the connection; the last
> > part contains the "real response"; of course the client will have to look
> > for the parts to get information about processing/result
> > 
> > - or a solution with multiple requests during processing (every x
> > second/minute) until the response is generated
> > 
> >> -----Ursprüngliche Nachricht-----
> >> Von: Daniel Lapadat [mailto:Daniel.Lapadat@baesystems.com]
> >> Gesendet: Mittwoch, 8. November 2006 20:13
> >> An: httpclient-user@jakarta.apache.org
> >> Betreff: httpclient proxy timeouts with long responses
> >>
> >> I have not been able to find a solution to the following problem.
> >>
> >> We have an application that talks to servlets using HTTPClient.  It
> >> requests information from the servlets, sometimes the response can take
> >> a long time ( > 10 minutes ).  Everything was working fine until some
> >> of our new users sat behind a proxy server.  The proxy server is timing
> >> out our HTTP connection at around 5 minutes.  We of course cannot
> >> modify any of the proxy server's settings.  Is there some way of
> >> sending a keep alive signal to the proxy server?
> >>  I thought that HTTP 1.1 automatically did this.
> >>
> >> The only solution we have found is to start writing white space back to
> >> the client every couple of seconds but if we do that and there is an
> >> error on the server side, the client will not get any response back
> >> from the server because we started to write to the output stream of the
> >> HTTPResponse object.  sendError causes an IllegalStateException on the
> >> server end.  In order to avoid this we cannot write to the output
> >> stream until we have a response from the database or other systems.
> >>
> >> Any help would be appreciated,
> >>
> >> Dan
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
> >> For additional commands, e-mail: httpclient-user-
> >> help@jakarta.apache.org
> > 
> > 
> > 
> > 
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
> > For additional commands, e-mail: httpclient-user-help@jakarta.apache.org
> > 
> 


---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-user-help@jakarta.apache.org


Re: AW: httpclient proxy timeouts with long responses

Posted by Daniel Lapadat <Da...@baesystems.com>.
Breu,

Thanks for the ideas.

We are already using the first idea but have a problem if an error happens.  We
cannot notify the client that the error has happened because we have already
started to write to the stream.  The client now does not know what happened and
will process everything like normal, in which case they get an empty string as
the response, which results in a failure (expecting XML).  On the server end we
will get an IllegalStateException because we have already committed.  Also I
believe that to use a multipart response I would need to have all of the data
upfront, at least that is what it looks like in the API documentation for
HTTPClient.

If I understand you, you are suggesting to take an asynchronous approach, which
I would like to avoid as it would involve a rewrite.  You are suggesting that we
basically poll the server for when our information is done, right?

Dan

Konstantin Breu wrote:
> 2 ideas:
> 
> - kind of http push: send a multipart response; the first parts are sent
> every x second/minute just to make the proxy keep the connection; the last
> part contains the "real response"; of course the client will have to look
> for the parts to get information about processing/result
> 
> - or a solution with multiple requests during processing (every x
> second/minute) until the response is generated
> 
>> -----Ursprüngliche Nachricht-----
>> Von: Daniel Lapadat [mailto:Daniel.Lapadat@baesystems.com]
>> Gesendet: Mittwoch, 8. November 2006 20:13
>> An: httpclient-user@jakarta.apache.org
>> Betreff: httpclient proxy timeouts with long responses
>>
>> I have not been able to find a solution to the following problem.
>>
>> We have an application that talks to servlets using HTTPClient.  It
>> requests information from the servlets, sometimes the response can take
>> a long time ( > 10 minutes ).  Everything was working fine until some
>> of our new users sat behind a proxy server.  The proxy server is timing
>> out our HTTP connection at around 5 minutes.  We of course cannot
>> modify any of the proxy server's settings.  Is there some way of
>> sending a keep alive signal to the proxy server?
>>  I thought that HTTP 1.1 automatically did this.
>>
>> The only solution we have found is to start writing white space back to
>> the client every couple of seconds but if we do that and there is an
>> error on the server side, the client will not get any response back
>> from the server because we started to write to the output stream of the
>> HTTPResponse object.  sendError causes an IllegalStateException on the
>> server end.  In order to avoid this we cannot write to the output
>> stream until we have a response from the database or other systems.
>>
>> Any help would be appreciated,
>>
>> Dan
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
>> For additional commands, e-mail: httpclient-user-
>> help@jakarta.apache.org
> 
> 
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: httpclient-user-help@jakarta.apache.org
> 

-- 
Daniel Lapadat
Software Engineer
BAE Systems -- National Security Solutions
6310 Hillside Court
Suite 200
Columbia, MD 21046
Office: 410-309-9366 x 2096
Fax:    410-309-9370

---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-user-help@jakarta.apache.org


AW: httpclient proxy timeouts with long responses

Posted by Konstantin Breu <Ko...@gmx.net>.
2 ideas:

- kind of http push: send a multipart response; the first parts are sent
every x second/minute just to make the proxy keep the connection; the last
part contains the "real response"; of course the client will have to look
for the parts to get information about processing/result

- or a solution with multiple requests during processing (every x
second/minute) until the response is generated

> -----Ursprüngliche Nachricht-----
> Von: Daniel Lapadat [mailto:Daniel.Lapadat@baesystems.com]
> Gesendet: Mittwoch, 8. November 2006 20:13
> An: httpclient-user@jakarta.apache.org
> Betreff: httpclient proxy timeouts with long responses
> 
> I have not been able to find a solution to the following problem.
> 
> We have an application that talks to servlets using HTTPClient.  It
> requests information from the servlets, sometimes the response can take
> a long time ( > 10 minutes ).  Everything was working fine until some
> of our new users sat behind a proxy server.  The proxy server is timing
> out our HTTP connection at around 5 minutes.  We of course cannot
> modify any of the proxy server's settings.  Is there some way of
> sending a keep alive signal to the proxy server?
>  I thought that HTTP 1.1 automatically did this.
> 
> The only solution we have found is to start writing white space back to
> the client every couple of seconds but if we do that and there is an
> error on the server side, the client will not get any response back
> from the server because we started to write to the output stream of the
> HTTPResponse object.  sendError causes an IllegalStateException on the
> server end.  In order to avoid this we cannot write to the output
> stream until we have a response from the database or other systems.
> 
> Any help would be appreciated,
> 
> Dan
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: httpclient-user-
> help@jakarta.apache.org




---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-user-help@jakarta.apache.org


Re: httpclient proxy timeouts with long responses

Posted by Oleg Kalnichevski <ol...@apache.org>.
On Wed, 2006-11-08 at 15:16 -0500, Daniel Lapadat wrote:
> Oleg,
> 
> It takes a long time to generate the response.  The response content may be
> small in size.
> 
> Dan
> 

Daniel,

I am afraid you are out of luck. 10 min latency is really extreme. The
proxy server assumes the connection that stays idle for so long is
likely to be stale and just drops it. There are no provisions in the
HTTP spec I know of to inform the proxy that the connection is still
being used.

HTTP is simply not the right protocol for such scenarios.

Oleg 


> Oleg Kalnichevski wrote:
> > On Wed, 2006-11-08 at 14:13 -0500, Daniel Lapadat wrote:
> >> I have not been able to find a solution to the following problem.
> >>
> >> We have an application that talks to servlets using HTTPClient.  It requests
> >> information from the servlets, sometimes the response can take a long time ( >
> >> 10 minutes ). 
> > 
> > Daniel,
> > 
> > Just to make sure we all understand you right. Does it take > 10 min to
> > stream out response really large response content or does it take such a
> > long while to generate a response (high latency scenario)?
> > 
> > Oleg
> > 
> >>  Everything was working fine until some of our new users sat
> >> behind a proxy server.  The proxy server is timing out our HTTP connection at
> >> around 5 minutes.  We of course cannot modify any of the proxy server's
> >> settings.  Is there some way of sending a keep alive signal to the proxy server?
> >>  I thought that HTTP 1.1 automatically did this.
> >>
> >> The only solution we have found is to start writing white space back to the
> >> client every couple of seconds but if we do that and there is an error on the
> >> server side, the client will not get any response back from the server because
> >> we started to write to the output stream of the HTTPResponse object.  sendError
> >> causes an IllegalStateException on the server end.  In order to avoid this we
> >> cannot write to the output stream until we have a response from the database or
> >> other systems.
> >>
> >> Any help would be appreciated,
> >>
> >> Dan
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
> >> For additional commands, e-mail: httpclient-user-help@jakarta.apache.org
> >>
> >>
> > 
> > 
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
> > For additional commands, e-mail: httpclient-user-help@jakarta.apache.org
> > 
> 


---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-user-help@jakarta.apache.org


Re: httpclient proxy timeouts with long responses

Posted by Daniel Lapadat <Da...@baesystems.com>.
Oleg,

It takes a long time to generate the response.  The response content may be
small in size.

Dan

Oleg Kalnichevski wrote:
> On Wed, 2006-11-08 at 14:13 -0500, Daniel Lapadat wrote:
>> I have not been able to find a solution to the following problem.
>>
>> We have an application that talks to servlets using HTTPClient.  It requests
>> information from the servlets, sometimes the response can take a long time ( >
>> 10 minutes ). 
> 
> Daniel,
> 
> Just to make sure we all understand you right. Does it take > 10 min to
> stream out response really large response content or does it take such a
> long while to generate a response (high latency scenario)?
> 
> Oleg
> 
>>  Everything was working fine until some of our new users sat
>> behind a proxy server.  The proxy server is timing out our HTTP connection at
>> around 5 minutes.  We of course cannot modify any of the proxy server's
>> settings.  Is there some way of sending a keep alive signal to the proxy server?
>>  I thought that HTTP 1.1 automatically did this.
>>
>> The only solution we have found is to start writing white space back to the
>> client every couple of seconds but if we do that and there is an error on the
>> server side, the client will not get any response back from the server because
>> we started to write to the output stream of the HTTPResponse object.  sendError
>> causes an IllegalStateException on the server end.  In order to avoid this we
>> cannot write to the output stream until we have a response from the database or
>> other systems.
>>
>> Any help would be appreciated,
>>
>> Dan
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
>> For additional commands, e-mail: httpclient-user-help@jakarta.apache.org
>>
>>
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: httpclient-user-help@jakarta.apache.org
> 

-- 
Daniel Lapadat
Software Engineer
BAE Systems -- National Security Solutions
6310 Hillside Court
Suite 200
Columbia, MD 21046
Office: 410-309-9366 x 2096
Fax:    410-309-9370

---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-user-help@jakarta.apache.org


Re: httpclient proxy timeouts with long responses

Posted by Oleg Kalnichevski <ol...@apache.org>.
On Wed, 2006-11-08 at 14:13 -0500, Daniel Lapadat wrote:
> I have not been able to find a solution to the following problem.
> 
> We have an application that talks to servlets using HTTPClient.  It requests
> information from the servlets, sometimes the response can take a long time ( >
> 10 minutes ). 

Daniel,

Just to make sure we all understand you right. Does it take > 10 min to
stream out response really large response content or does it take such a
long while to generate a response (high latency scenario)?

Oleg

>  Everything was working fine until some of our new users sat
> behind a proxy server.  The proxy server is timing out our HTTP connection at
> around 5 minutes.  We of course cannot modify any of the proxy server's
> settings.  Is there some way of sending a keep alive signal to the proxy server?
>  I thought that HTTP 1.1 automatically did this.
> 
> The only solution we have found is to start writing white space back to the
> client every couple of seconds but if we do that and there is an error on the
> server side, the client will not get any response back from the server because
> we started to write to the output stream of the HTTPResponse object.  sendError
> causes an IllegalStateException on the server end.  In order to avoid this we
> cannot write to the output stream until we have a response from the database or
> other systems.
> 
> Any help would be appreciated,
> 
> Dan
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: httpclient-user-help@jakarta.apache.org
> 
> 


---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-user-help@jakarta.apache.org


Re: httpclient proxy timeouts with long responses

Posted by Daniel Lapadat <Da...@baesystems.com>.
Jeff,

How would I use it?

Would the client see the response or would HTTPClient handle it for me?  Would I
need to send it every 4 minutes so that the proxy does not end the connection?
The client would be expecting to parse the response and in this case would have
nothing to parse.  Is this to handle the search in a asynchronous manner?  If
so, I would like to avoid that if possible.  Thank you,

Dan

Segal, Jeffrey wrote:
> Dan,
> 
> Are you using the Expect-Continue header in your requests?  If not, that
> may be what you need.
> 
> Jeff 
> 
> -----Original Message-----
> From: Daniel Lapadat [mailto:Daniel.Lapadat@baesystems.com] 
> Sent: Wednesday, November 08, 2006 2:13 PM
> To: httpclient-user@jakarta.apache.org
> Subject: httpclient proxy timeouts with long responses
> 
> I have not been able to find a solution to the following problem.
> 
> We have an application that talks to servlets using HTTPClient.  It
> requests information from the servlets, sometimes the response can take
> a long time ( > 10 minutes ).  Everything was working fine until some of
> our new users sat behind a proxy server.  The proxy server is timing out
> our HTTP connection at around 5 minutes.  We of course cannot modify any
> of the proxy server's settings.  Is there some way of sending a keep
> alive signal to the proxy server?
>  I thought that HTTP 1.1 automatically did this.
> 
> The only solution we have found is to start writing white space back to
> the client every couple of seconds but if we do that and there is an
> error on the server side, the client will not get any response back from
> the server because we started to write to the output stream of the
> HTTPResponse object.  sendError causes an IllegalStateException on the
> server end.  In order to avoid this we cannot write to the output stream
> until we have a response from the database or other systems.
> 
> Any help would be appreciated,
> 
> Dan
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: httpclient-user-help@jakarta.apache.org
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: httpclient-user-help@jakarta.apache.org
> 

-- 
Daniel Lapadat
Software Engineer
BAE Systems -- National Security Solutions
6310 Hillside Court
Suite 200
Columbia, MD 21046
Office: 410-309-9366 x 2096
Fax:    410-309-9370

---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-user-help@jakarta.apache.org


RE: httpclient proxy timeouts with long responses

Posted by "Segal, Jeffrey" <Je...@solers.com>.
Dan,

Are you using the Expect-Continue header in your requests?  If not, that
may be what you need.

Jeff 

-----Original Message-----
From: Daniel Lapadat [mailto:Daniel.Lapadat@baesystems.com] 
Sent: Wednesday, November 08, 2006 2:13 PM
To: httpclient-user@jakarta.apache.org
Subject: httpclient proxy timeouts with long responses

I have not been able to find a solution to the following problem.

We have an application that talks to servlets using HTTPClient.  It
requests information from the servlets, sometimes the response can take
a long time ( > 10 minutes ).  Everything was working fine until some of
our new users sat behind a proxy server.  The proxy server is timing out
our HTTP connection at around 5 minutes.  We of course cannot modify any
of the proxy server's settings.  Is there some way of sending a keep
alive signal to the proxy server?
 I thought that HTTP 1.1 automatically did this.

The only solution we have found is to start writing white space back to
the client every couple of seconds but if we do that and there is an
error on the server side, the client will not get any response back from
the server because we started to write to the output stream of the
HTTPResponse object.  sendError causes an IllegalStateException on the
server end.  In order to avoid this we cannot write to the output stream
until we have a response from the database or other systems.

Any help would be appreciated,

Dan

---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-user-help@jakarta.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-user-help@jakarta.apache.org